{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Actor Critic Policy Optimization with PyTorch\n",
    "\n",
    "In this notebook, we will look at policy optimization using PyTorch. Let us quickly go through the derivation of what is policy Optimization and then we will apply this to CartPole environment - just like what we did in Chapter 6. \n",
    "\n",
    "\n",
    "### Derivation of Policy Gradient\n",
    "In Policy Optimization, we will have a neural network which takes in state `s` as input and produces the `logits` for action probabilities. \n",
    "\n",
    "The policy is parameterized by $\\theta$\n",
    "$$\\pi_\\theta(a|s)$$\n",
    "\n",
    "The agent follows the policy and generates the trajectory $\\large \\tau$ \n",
    "\n",
    "$$ s_1 \\rightarrow a_1 \\rightarrow s_2 \\rightarrow a_2 \\rightarrow .... \\rightarrow s_{T-1} \\rightarrow a_{T-1} \\rightarrow s_T \\rightarrow a_T$$ \n",
    "\n",
    "here $s_T$ is not necessarily the terminal state but some time horizon T upto which we are looking at the trajectory. \n",
    "\n",
    "The probability of trajectory $\\large \\tau$ depends on the transition probabilities $p(s_t+1 | s_t, a_t)$ and the policy $\\pi_\\theta(a_t|s_t)$. It is given by the expression:\n",
    "\n",
    "$$p_\\theta(\\tau) = p_\\theta(s_1, a_1, s_2, a_2, ..., s_T, a_T) = p(s_1)\\prod_{t=1}^{T}\\pi_\\theta(a_t|s_t)p(s_{t+1}|s_t,a_t)$$\n",
    "\n",
    "The expected return from following the policy $\\pi$ is given by:\n",
    "\n",
    "$$J(\\theta) = {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\sum_{t} \\gamma^t r(s_t, a_t) \\right]$$\n",
    "\n",
    "We want to find the $\\theta$ which maximizes the expected reward/return $J(\\theta)$. In other words, the optimal $\\theta=\\theta^*$ is given by expression\n",
    "\n",
    "$$\\theta^* = arg \\underset{\\theta}{max}{\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\sum_{t} \\gamma^t r(s_t, a_t) \\right] $$\n",
    "\n",
    "\n",
    "Moving on, let us try to find the optimal $\\theta$. To keep the notations easier to understand, we will replace $\\sum_{t} \\gamma^t r(s_t, a_t)$ as $r(\\tau)$:\n",
    "\n",
    "$$J(\\theta) = {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ r(\\tau) \\right] = \\int p_\\theta(\\tau)r(\\tau) d\\tau$$\n",
    "\n",
    "We take the gradient/derivative of above expression with respect to $\\theta$:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\nabla_{\\theta} \\int p_\\theta(\\tau)r(\\tau) d\\tau $$\n",
    "\n",
    "By linearity we can move the gradient inside the integral:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\int \\nabla_{\\theta} p_\\theta(\\tau)r(\\tau) d\\tau $$\n",
    "\n",
    "Using log derivative trick, we know that $\\nabla_x f(x) = f(x) \\nabla_x \\log{f(x)}$. Using this we can write above expression as:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\int p_\\theta(\\tau) \\left[ \\nabla_{\\theta}\\log{p_\\theta(\\tau)} r(\\tau) \\right] d\\tau $$\n",
    "\n",
    "We can now write the integral back as expectation, which gives us the expression:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\nabla_{\\theta}\\log{p_\\theta(\\tau)} r(\\tau) \\right] $$\n",
    "\n",
    "Let us now expand the term $\\nabla_{\\theta}\\log{p_\\theta(\\tau)}$ by writing out the full expression of $p_\\theta(\\tau)$. \n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\nabla_{\\theta} \\log{ \\left[ p(s_1) \\prod_{t=1}^{T}\\pi_\\theta(a_t|s_t)p(s_{t+1}|s_t,a_t)\\right]}$$\n",
    "\n",
    "We know that log of product of terms can be written as sum of log of terms i.e. \n",
    "\n",
    "$$\\log{\\prod_i f_i(x)} = \\sum_i log{f_i(x)}$$ \n",
    "\n",
    "Using the above substitution, we get:\n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\nabla_{\\theta} \\left[ log{p(s_1)} +  \\sum_{t=1}^{T} \\left\\{ \\log{ \\pi_\\theta(a_t|s_t)} + \\log{p(s_{t+1}|s_t,a_t)} \\right\\} \\right]$$\n",
    "\n",
    "The only term dependent on $\\theta$ is $\\pi_\\theta(a_t|s_t)$. The other two terms $log{p(s_1)}$ and $\\log{p(s_{t+1}|s_t,a_t)}$ do not depend on $\\theta$. Accordingly, we can simplify the above expression as:\n",
    "\n",
    "$$ \\nabla_{\\theta}\\log{p_\\theta(\\tau)}  = \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t|s_t)} $$\n",
    "\n",
    "\n",
    "Substituting the above term into the expression for $\\nabla_{\\theta} J(\\theta)$, as well as expanding $r(\\tau)$ we get:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  {\\large E}_{\\tau \\sim p_\\theta(\\tau)} \\left[ \\left( \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t|s_t)} \\right) \\left( \\sum_{t=1}^{T} \\gamma^t r(s_t, a_t) \\right) \\right] $$\n",
    "\n",
    "We can now replace the outer expectation with an estimate over multiple trajectories to get the following expression for the gradient of policy objective:\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\frac{1}{N} \\sum_{i=1}^{N} \\left[ \\left( \\sum_{t=1}^{T} \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\right) \\left( \\sum_{t=1}^{T} \\gamma^t r(s_t^i, a_t^i) \\right) \\right] $$\n",
    "\n",
    "where i denotes the $i^{th}$ trajectory. \n",
    "\n",
    "To improve the policy, we now take a +ve step in $\\theta$ in the direction of $\\nabla_{\\theta} J(\\theta)$:\n",
    "\n",
    "$$\\theta = \\theta + \\alpha \\nabla_{\\theta} J(\\theta)$$\n",
    "\n",
    "To summarize, we design a model which takes state $s$ as input and produces the policy distribution $\\pi_\\theta(a|s)$ as the output of the model. We use a policy to generate returns and then change the model parameter $\\theta$ using the expression: $\\theta = \\theta + \\alpha \\nabla_{\\theta} J(\\theta)$\n",
    "\n",
    "\n",
    "### Rewards to Go Trick\n",
    "\n",
    "\n",
    "we drop the reward terms that came before time t as at time t, the action we take can only impact the reward which comes at time t and later. This leads to changing the 2nd inner sum going from t’=t to T instead of earlier sum over t’ going from t’=1 to T. i.e. the start index is now t’=t and not t=1. The revised expression is given below:\n",
    "\n",
    "\n",
    "$$\\nabla_{\\theta} J(\\theta) =  \\frac{1}{N} \\sum_{i=1}^{N} \\left[  \\sum_{t=1}^{T}  \\left( \\nabla_{\\theta} \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\sum_{t'=t}^{T} \\gamma^{t'-t} r(s_{t'}^i, a_{t'}^i) \\right) \\right] $$\n",
    "\n",
    "\n",
    "### Baseline Trick\n",
    "We can further reduce the variance using base line to get:\n",
    "\n",
    "$$\\nabla_\\theta J\\left(\\theta\\right)=\\frac{1}{N}\\sum_{i=1}^{N}\\sum_{t=1}^{T}{\\nabla_\\theta\\log{\\pi_\\theta\\left(a_t^i\\middle| s_t^i\\right)\\ }\\left[\\hat{Q}(s_t^i,\\ a_t^i) - b\\left(s_t^i\\right)\\right]\\\\}$$\n",
    "\n",
    "where:\n",
    "\n",
    "$$\\hat{Q}(s_t^i,\\ a_t^i)=\\ \\sum_{t^\\prime=t}^{T} \\gamma^{t'-t} r\\left(s_{t^\\prime}^i,a_{t^\\prime}^i\\right)$$\n",
    "\n",
    "\n",
    "### Actor Critic\n",
    "In Actor Critic, we fit the baseline to an estimator for state value V. We use a model as given below:\n",
    "\n",
    "\n",
    "![Actor Critic](./images/actor_critic.png \"Actor Critic\")\n",
    "\n",
    "\n",
    "\n",
    "The final update rule under Actor Critic is given below:\n",
    "\n",
    "$$\\nabla_{\\theta,\\phi} J\\left(\\theta,\\phi \\right)=\\frac{1}{N}\\sum_{i=1}^{N}\\sum_{t=1}^{T}{\\nabla_\\theta\\log{\\pi_\\theta\\left(a_t^i\\middle| s_t^i\\right)\\ }\\left[ \\hat{Q}(s_t^i,\\ a_t^i) -\\ V_\\phi(s_t^i)\\right]\\ }$$\n",
    "\n",
    "\n",
    "### Implementing Loss and Gradient Step in PyTorch/TensorFlow\n",
    "\n",
    "We will implement a pseudo loss function, whose derivative will give us $\\nabla_{\\theta,\\phi} J\\left(\\theta,\\phi \\right)$. Also as PyTorch/TensorFlow carryout a gradient Step, we will convert maximization to minimization by changing the sign of this objective function\n",
    "\n",
    "$$L_{CrossEntropy}(\\theta, \\phi) = - J(\\theta, \\phi) = - \\frac{1}{N} \\sum_{i=1}^{N}  \\sum_{t=1}^{T} \\left( \\log{\\pi_\\theta (a_t^i | s_t^i) } \\left[ \\hat{Q}(s_t^i,\\ a_t^i) - V_\\phi(s_t^i)\\right] \\right)$$\n",
    "\n",
    "To summarize, we will pass the state `s` through the network to get $\\log{ \\pi_\\theta(a_t^i|s_t^i)}$ and $V_\\phi(s_t)$. We will calculate the cross_entropy loss for the actions actually seen in the trajectory. We will then calculate the weighted mean of these individual loss terms in the trajectory with weights being the Advantage $\\hat{A}(s_t^i,\\ a_t^i) = \\hat{Q}(s_t^i,\\ a_t^i) - V_\\phi(s_t^i)$\n",
    "\n",
    "This will be followed by a gradient step in -ve direction of weighted NLL (negative log loss) i.e. in positive direction of the gradient of $J(\\theta, \\phi)= - L_{CrossEntropy}(\\theta, \\phi)$ \n",
    "\n",
    "We also add a regularization term known as Entropy. Entropy of a distribution is defined as:\n",
    "\n",
    "$$H(X) = \\sum_x -p(x).log(p(x))$$\n",
    "\n",
    "To keep enough exploration, we will want the probability to have a spread out distribution and not let the probability distribution to collapse to a single value or a small region too soon. Bigger the spread of a distribution, higher the entropy H(x) of a distribution. Accordingly, the term fed into PyTorch/TensorFlow minimizer is:\n",
    "\n",
    "\n",
    "$$Loss(\\theta, \\phi) = - J(\\theta, \\phi) - H(\\pi_\\theta(a_t^i|s_t^i)) = - \\frac{1}{N} \\sum_{i=1}^{N} \\left[ \\sum_{t=1}^{T} \\left(  \\log{ \\pi_\\theta(a_t^i|s_t^i)} \\left[ \\hat{Q}(s_t^i,\\ a_t^i) - V_\\phi(s_t^i)\\right] \\right) - \\beta \\sum_a \\pi_\\theta(a|s_t^i).\\log{ \\pi_\\theta(a|s_t^i)} \\right] $$\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "import gym\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.signal import convolve, gaussian\n",
    "\n",
    "import os\n",
    "import io\n",
    "import base64\n",
    "import time\n",
    "import glob\n",
    "from IPython.display import HTML\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Environment - CartPole \n",
    "\n",
    "We can use the setup here to run on any environment which has state as a single vector and actions are discrete. We will build it on Cart Pole and they try to run this on many other environments like Atari games and others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_env(env_name, seed=None):\n",
    "    # remove time limit wrapper from environment\n",
    "    env = gym.make(env_name).unwrapped\n",
    "    if seed is not None:\n",
    "        env.seed(seed)\n",
    "    return env"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAATSElEQVR4nO3dbYyd5Z3f8e+PwRgC2Y2pB+r4AbxZJ1tId81qxKaiDzSha5emdfIiK0dqxAsk54UjBXWVFnalbiLV0rbaJNtITSRSEFY2iWMpYbFS2i6wiZJIFGNYQ2wMixO8sWMHOxDCwy7GHv/7Ym4vx/aM53geOL5mvh/p6Nznf1/3uf8Xsn++ueY+Z1JVSJLaccGgG5AknRuDW5IaY3BLUmMMbklqjMEtSY0xuCWpMbMW3EnWJnkmyd4kt8/WeSRpvsls3MedZAj4a+BfAQeAR4GPVtVTM34ySZpnZuuK+3pgb1X9uKreALYA62bpXJI0r1w4S++7FNjf8/oA8DsTDV68eHFdffXVs9SKJLVn3759/PznP894+2YruMc72SlrMkk2ABsAVqxYwY4dO2apFUlqz8jIyIT7Zmup5ACwvOf1MuBg74CqurOqRqpqZHh4eJbakKS5Z7aC+1FgVZKVSS4C1gPbZulckjSvzMpSSVUdT/IJ4P8CQ8DdVbV7Ns4lSfPNbK1xU1X3A/fP1vtL0nzlJyclqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDVmWr+6LMk+4BVgFDheVSNJLge+AVwN7AN+r6p+Mb02JUknzcQV97+sqtVVNdK9vh14qKpWAQ91ryVJM2Q2lkrWAZu77c3Ah2bhHJI0b003uAv4iySPJdnQ1a6sqkMA3fMV0zyHJKnHtNa4gRuq6mCSK4AHkjzd74Fd0G8AWLFixTTbkKT5Y1pX3FV1sHs+DNwLXA88n2QJQPd8eIJj76yqkaoaGR4enk4bkjSvTDm4k1ya5O0nt4HfBXYB24BbumG3APdNt0lJ0pums1RyJXBvkpPv87Wq+j9JHgW2JrkV+Anwkem3KUk6acrBXVU/Bn5rnPoLwAem05QkaWJ+clKSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqzKTBneTuJIeT7OqpXZ7kgSTPds+LevbdkWRvkmeSrJmtxiVpvurnivseYO1ptduBh6pqFfBQ95ok1wDrgWu7Y76YZGjGupUkTR7cVfU94MXTyuuAzd32ZuBDPfUtVXW0qp4D9gLXz0yrkiSY+hr3lVV1CKB7vqKrLwX294w70NXOkGRDkh1Jdhw5cmSKbUjS/DPTP5zMOLUab2BV3VlVI1U1Mjw8PMNtSNLcNdXgfj7JEoDu+XBXPwAs7xm3DDg49fYkSaebanBvA27ptm8B7uupr0+yMMlKYBWwfXotSpJ6XTjZgCRfB24EFic5APwR8MfA1iS3Aj8BPgJQVbuTbAWeAo4DG6tqdJZ6l6R5adLgrqqPTrDrAxOM3wRsmk5TkqSJ+clJSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNmTS4k9yd5HCSXT21Tyf5aZKd3ePmnn13JNmb5Jkka2arcUmar/q54r4HWDtO/fNVtbp73A+Q5BpgPXBtd8wXkwzNVLOSpD6Cu6q+B7zY5/utA7ZU1dGqeg7YC1w/jf4kSaeZzhr3J5I82S2lLOpqS4H9PWMOdLUzJNmQZEeSHUeOHJlGG5I0v0w1uL8EvAtYDRwCPtvVM87YGu8NqurOqhqpqpHh4eEptiFJ88+Ugruqnq+q0ao6AXyZN5dDDgDLe4YuAw5Or0VJUq8pBXeSJT0vPwycvONkG7A+ycIkK4FVwPbptShJ6nXhZAOSfB24EVic5ADwR8CNSVYztgyyD/g4QFXtTrIVeAo4DmysqtFZ6VyS5qlJg7uqPjpO+a6zjN8EbJpOU5KkifnJSUlqjMEtSY0xuCWpMQa3JDXG4Jakxhjc0gSOH/1bXjv8HFXjfvhXGphJbweU5qvXf3GQZ+//71z2D3/9lPqyf/J7XLJoyQRHSbPP4JbO4sTxN3j5wFOn1Ebf+NsBdSONcalEkhpjcEtSYwxuSWqMwS1N4PCuvzyjdtmSd3PJ5csG0I30JoNbmsAbr/3ijNqFC9/G0IKFA+hGepPBLUmNMbglqTEGtyQ1xuCWpMYY3JLUmEmDO8nyJN9JsifJ7iSf7OqXJ3kgybPd86KeY+5IsjfJM0nWzOYEJGm+6eeK+zjw+1X1j4D3ARuTXAPcDjxUVauAh7rXdPvWA9cCa4EvJhmajeYlaT6aNLir6lBVPd5tvwLsAZYC64DN3bDNwIe67XXAlqo6WlXPAXuB62e4b0mat85pjTvJ1cB1wCPAlVV1CMbCHbiiG7YU2N9z2IGudvp7bUiyI8mOI0eOTKF1SZqf+g7uJJcB3wRuq6qXzzZ0nNoZ30RfVXdW1UhVjQwPD/fbhiTNe30Fd5IFjIX2V6vqW135+SRLuv1LgMNd/QCwvOfwZcDBmWlXemsc+7tXGD36d2fUF/7KFeOMlt5a/dxVEuAuYE9Vfa5n1zbglm77FuC+nvr6JAuTrARWAdtnrmVp9r32/I94/aVDpxYTFv/GDYNpSOrRz2/AuQH4GPDDJDu72h8AfwxsTXIr8BPgIwBVtTvJVuApxu5I2VhVozPduCTNV5MGd1X9gPHXrQE+MMExm4BN0+hLkjQBPzkpSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbil01Sd4Mie759R/9UV/5gFly4a5wjprWVwS6crOPrymd8Rv+Btv8rQgoUDaEg6lcEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1Jj+vllwcuTfCfJniS7k3yyq386yU+T7OweN/ccc0eSvUmeSbJmNicgSfNNP78s+Djw+1X1eJK3A48leaDb9/mq+pPewUmuAdYD1wLvBB5M8m5/YbAkzYxJr7ir6lBVPd5tvwLsAZae5ZB1wJaqOlpVzwF7getnollJ0jmucSe5GrgOeKQrfSLJk0nuTnLySxyWAvt7DjvA2YNeOq8cf/0VavTYoNuQJtR3cCe5DPgmcFtVvQx8CXgXsBo4BHz25NBxDq9x3m9Dkh1Jdhw5cub3QkiD8tLfPMEbr754Si1DF7Lo10YG1JF0qr6CO8kCxkL7q1X1LYCqer6qRqvqBPBl3lwOOQAs7zl8GXDw9PesqjuraqSqRoaHh6czB2nWJUNcsuidg25DAvq7qyTAXcCeqvpcT31Jz7APA7u67W3A+iQLk6wEVgHbZ65lSZrf+rmr5AbgY8APk+zsan8AfDTJasaWQfYBHweoqt1JtgJPMXZHykbvKJGkmTNpcFfVDxh/3fr+sxyzCdg0jb4kSRPwk5OS1BiDW5IaY3BLUmMMbklqjMEtSY0xuCWpMQa3JDXG4JZ6VBXHX3/tjPrQwreRC/zrovODfxKlHieOv8GR3d85o774N25gaOGlA+hIOpPBLZ2m6owvswTC2Nf2SINncEtSYwxuSWqMwS1Jjenna12lpp04cYLbbruN/fv3Tzp2wVDY+C8u57KFQ6fUt2zZwvf/y119nW/jxo3cdNNNU+pV6ofBrTmvqnjwwQfZs2fPpGMvvuhCbv2d9Vy0YBFVY/9DeuEFb7Bnzx7+/H893tf5PvjBD06rX2kyBrd0ml8cu5KdR/4tx2ohAEsufo4T9eiAu5LeZHBLPUbrQna+dCOXXHrZ39d+9vpV/PLY4gF2JZ3KH05Kpxmti055fbwWcuTosgF1I52pn18WfHGS7UmeSLI7yWe6+uVJHkjybPe8qOeYO5LsTfJMkjWzOQFpJoXi4gtePaW2IK+z9JK9A+pIOlM/V9xHgfdX1W8Bq4G1Sd4H3A48VFWrgIe61yS5BlgPXAusBb6YZGi8N5bON+E4Vxz7Fi+/uIdXfrmfS4deYuWlu3jb0CuDbk36e/38suACTl6CLOgeBawDbuzqm4HvAv+pq2+pqqPAc0n2AtcDD090jmPHjvGzn/1sajOQJnHixAlGR0f7Gnv02Ci3/enXKL7O5b9yCf/sN68iFHv+5kjf53v55Zf986xpO3bs2IT7+vrhZHfF/Bjw68D/qKpHklxZVYcAqupQkiu64UuB/9dz+IGuNqEXXniBr3zlK/20Ip2zquKll17qe/yJKqB44Zev8efff+qcz/fwww9z/Pjxcz5O6vXCCy9MuK+v4K6qUWB1kncA9yZ571mGj/dNPGd8a0+SDcAGgBUrVvCpT32qn1akczY6Oso999zD4cOH35LzrVmzhltvvfUtOZfmrm984xsT7junu0qq6iXGlkTWAs8nWQLQPZ/8W3EAWN5z2DLg4DjvdWdVjVTVyPDw8Lm0IUnzWj93lQx3V9okuQS4CXga2Abc0g27Bbiv294GrE+yMMlKYBWwfYb7lqR5q5+lkiXA5m6d+wJga1V9O8nDwNYktwI/AT4CUFW7k2wFngKOAxu7pRZJ0gzo566SJ4Hrxqm/AHxggmM2AZum3Z0k6Qx+clKSGmNwS1Jj/JIpzXlJuOmmm3jPe97zlpzvqquuekvOo/nL4Nacd8EFF/CFL3xh0G1IM8alEklqjMEtSY0xuCWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUmH5+WfDFSbYneSLJ7iSf6eqfTvLTJDu7x809x9yRZG+SZ5Ksmc0JSNJ808/3cR8F3l9VryZZAPwgyf/u9n2+qv6kd3CSa4D1wLXAO4EHk7zbXxgsSTNj0ivuGvNq93JB96izHLIO2FJVR6vqOWAvcP20O5UkAX2ucScZSrITOAw8UFWPdLs+keTJJHcnWdTVlgL7ew4/0NUkSTOgr+CuqtGqWg0sA65P8l7gS8C7gNXAIeCz3fCM9xanF5JsSLIjyY4jR45MoXVJmp/O6a6SqnoJ+C6wtqqe7wL9BPBl3lwOOQAs7zlsGXBwnPe6s6pGqmpkeHh4Kr1L0rzUz10lw0ne0W1fAtwEPJ1kSc+wDwO7uu1twPokC5OsBFYB22e0a0max/q5q2QJsDnJEGNBv7Wqvp3kK0lWM7YMsg/4OEBV7U6yFXgKOA5s9I4SSZo5kwZ3VT0JXDdO/WNnOWYTsGl6rUmSxuMnJyWpMQa3JDXG4JakxhjcktQYg1uSGmNwS1JjDG5JaozBLUmNMbglqTEGtyQ1xuCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4JakxBrckNcbglqTGGNyS1BiDW5IaY3BLUmNSVYPugSRHgNeAnw+6l1mwGOfVmrk6N+fVlquqani8HedFcAMk2VFVI4PuY6Y5r/bM1bk5r7nDpRJJaozBLUmNOZ+C+85BNzBLnFd75urcnNcccd6scUuS+nM+XXFLkvow8OBOsjbJM0n2Jrl90P2cqyR3JzmcZFdP7fIkDyR5tnte1LPvjm6uzyRZM5iuJ5dkeZLvJNmTZHeST3b1pueW5OIk25M80c3rM1296XmdlGQoyV8l+Xb3eq7Ma1+SHybZmWRHV5sTc5uSqhrYAxgCfgT8GnAR8ARwzSB7msIc/jnw28Cuntp/A27vtm8H/mu3fU03x4XAym7uQ4OewwTzWgL8drf9duCvu/6bnhsQ4LJuewHwCPC+1ufVM7//AHwN+PZc+bPY9bsPWHxabU7MbSqPQV9xXw/sraofV9UbwBZg3YB7OidV9T3gxdPK64DN3fZm4EM99S1VdbSqngP2Mvbf4LxTVYeq6vFu+xVgD7CUxudWY17tXi7oHkXj8wJIsgz4N8D/7Ck3P6+zmMtzO6tBB/dSYH/P6wNdrXVXVtUhGAtA4Iqu3uR8k1wNXMfY1Wnzc+uWE3YCh4EHqmpOzAv4U+A/Aid6anNhXjD2j+tfJHksyYauNlfmds4uHPD5M05tLt/m0tx8k1wGfBO4rapeTsabwtjQcWrn5dyqahRYneQdwL1J3nuW4U3MK8kHgcNV9ViSG/s5ZJzaeTevHjdU1cEkVwAPJHn6LGNbm9s5G/QV9wFgec/rZcDBAfUyk55PsgSgez7c1Zuab5IFjIX2V6vqW115TswNoKpeAr4LrKX9ed0A/Lsk+xhbcnx/kj+j/XkBUFUHu+fDwL2MLX3MiblNxaCD+1FgVZKVSS4C1gPbBtzTTNgG3NJt3wLc11Nfn2RhkpXAKmD7APqbVMYure8C9lTV53p2NT23JMPdlTZJLgFuAp6m8XlV1R1Vtayqrmbs79FfVtW/p/F5ASS5NMnbT24DvwvsYg7MbcoG/dNR4GbG7lj4EfCHg+5nCv1/HTgEHGPsX/pbgX8APAQ82z1f3jP+D7u5PgP860H3f5Z5/VPG/vfySWBn97i59bkBvwn8VTevXcB/7upNz+u0Od7Im3eVND8vxu46e6J77D6ZE3NhblN9+MlJSWrMoJdKJEnnyOCWpMYY3JLUGINbkhpjcEtSYwxuSWqMwS1JjTG4Jakx/x83n7oJZALo5QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "env_name = 'CartPole-v1'\n",
    "\n",
    "env = make_env(env_name)\n",
    "env.reset()\n",
    "plt.imshow(env.render(\"rgb_array\"))\n",
    "state_shape, n_actions = env.observation_space.shape, env.action_space.n\n",
    "state_dim = state_shape[0]\n",
    "env.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build Actor Critic Network\n",
    "\n",
    "We will build two simple networks that take in state. One network produces logits for the action probabilities. 2nd network produces the Value of the state. The observation space and action space is as given below for CartPole\n",
    "\n",
    "    Observation:\n",
    "        Type: Box(4)\n",
    "        Num     Observation               Min                     Max\n",
    "        0       Cart Position             -4.8                    4.8\n",
    "        1       Cart Velocity             -Inf                    Inf\n",
    "        2       Pole Angle                -0.418 rad (-24 deg)    0.418 rad (24 deg)\n",
    "        3       Pole Angular Velocity     -Inf                    Inf\n",
    "    Actions:\n",
    "        Type: Discrete(2)\n",
    "        Num   Action\n",
    "        0     Push cart to the left\n",
    "        1     Push cart to the right\n",
    "        \n",
    "\n",
    "Each model will be a simple one with 1 hidden layer with Relu activation and final layer being logits (for policy/actor network) and value of the state for the Critic Network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "device(type='cpu')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "device"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ActorCritic(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(ActorCritic, self).__init__()\n",
    "        self.fc1 = nn.Linear(state_dim, 128)\n",
    "        self.actor = nn.Linear(128,n_actions)\n",
    "        self.critic = nn.Linear(128,1)\n",
    "\n",
    "\n",
    "    def forward(self, s):\n",
    "        x = F.relu(self.fc1(s))\n",
    "        logits = self.actor(x)\n",
    "        state_value = self.critic(x)\n",
    "        return logits, state_value\n",
    "        \n",
    "model = ActorCritic()\n",
    "model = model.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict Action Probabilities\n",
    "\n",
    "We will use this function to generate the trajectory. It will not be used for doing back propagation. So we will use PyTorch `no_grad()` to avoid gradient calculations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sample_action(state):\n",
    "    \"\"\"\n",
    "    params: states: [batch, state_dim]\n",
    "    returns: probs: [batch, n_actions]\n",
    "    \"\"\"\n",
    "    state = torch.tensor(state, device=device, dtype=torch.float32)\n",
    "    with torch.no_grad():\n",
    "        logits,_ = model(state)\n",
    "    action_probs = nn.functional.softmax(logits, -1).detach().numpy()[0]\n",
    "    action = np.random.choice(n_actions, p=action_probs)\n",
    "    return action"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Play game and generate Trajectory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_trajectory(env, n_steps=1000):\n",
    "    \"\"\"\n",
    "    Play a session and genrate a trajectory\n",
    "    returns: arrays of states, actions, rewards\n",
    "    \"\"\"\n",
    "    states, actions, rewards = [], [], []\n",
    "    \n",
    "    # initialize the environment\n",
    "    s = env.reset()\n",
    "    \n",
    "    #generate n_steps of trajectory:\n",
    "    for t in range(n_steps):\n",
    "        #sample action based on action_probs\n",
    "        a = sample_action(np.array([s]))\n",
    "        next_state, r, done, _ = env.step(a)\n",
    "        \n",
    "        #update arrays\n",
    "        states.append(s)\n",
    "        actions.append(a)\n",
    "        rewards.append(r)\n",
    "\n",
    "        s = next_state\n",
    "        if done:\n",
    "            break\n",
    "    \n",
    "    return states, actions, rewards"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate Rewards to Go\n",
    "\n",
    " $G(s_t) = \\sum_{t'=t}^{T} \\gamma^{t-t'} r(s_{t'}^i, a_{t'}^i)$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_rewards_to_go(rewards, gamma=0.99):\n",
    "    \n",
    "    T = len(rewards) # total number of individual rewards\n",
    "    # empty array to return the rewards to go\n",
    "    rewards_to_go = [0]*T \n",
    "    rewards_to_go[T-1] = rewards[T-1]\n",
    "    \n",
    "    for i in range(T-2, -1, -1): #go from T-2 to 0\n",
    "        rewards_to_go[i] = gamma * rewards_to_go[i+1] + rewards[i]\n",
    "    \n",
    "    return rewards_to_go"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train on one trajectory\n",
    "\n",
    "We will calculate the loss and take a gradient step. We will use Adam Optimizer\n",
    "\n",
    "**policy network loss:**\n",
    "\n",
    "We are taking only one trajectory. so N=1. We will, however, average it over the number of actions to get the average loss. So the function we will actually implement is as given below:\n",
    "\n",
    "\n",
    "$$Loss(\\theta, \\phi) = - J(\\theta, \\phi) - H(\\pi_\\theta(a_t|s_t)) = - \\frac{1}{T}  \\left[ \\sum_{t=1}^{T} \\left(  \\log{ \\pi_\\theta(a_t|s_t)} \\left[ \\hat{Q}(s_t,\\ a_t) - V_\\phi(s_t)\\right] \\right) - \\beta \\sum_{a} \\pi_\\theta(a|s_t).\\log{ \\pi_\\theta(a|s_t)} \\right] $$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#init Optimizer\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\n",
    "\n",
    "def train_one_episode(states, actions, rewards, gamma=0.99, entropy_coef=1e-2):\n",
    "    \n",
    "    # get rewards to go\n",
    "    rewards_to_go = get_rewards_to_go(rewards, gamma)\n",
    "\n",
    "    # convert numpy array to torch tensors\n",
    "    states = torch.tensor(states, device=device, dtype=torch.float)\n",
    "    actions = torch.tensor(actions, device=device, dtype=torch.long)\n",
    "    rewards_to_go = torch.tensor(rewards_to_go, device=device, dtype=torch.float)\n",
    "    \n",
    "\n",
    "    # get action probabilities from states\n",
    "    logits, state_values = model(states)\n",
    "    probs = nn.functional.softmax(logits, -1)\n",
    "    log_probs = nn.functional.log_softmax(logits, -1)\n",
    "    \n",
    "    log_probs_for_actions = log_probs[range(len(actions)), actions]\n",
    "    \n",
    "    advantage = rewards_to_go - state_values.squeeze(-1)\n",
    "    \n",
    "    #Compute loss to be minized\n",
    "    J = torch.mean(log_probs_for_actions*(advantage))\n",
    "    H = -(probs*log_probs).sum(-1).mean()\n",
    "    \n",
    "    loss = -(J+entropy_coef*H)\n",
    "\n",
    "    optimizer.zero_grad()\n",
    "    loss.backward()\n",
    "    optimizer.step()\n",
    "    \n",
    "    return np.sum(rewards) #to show progress on training\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train the agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mean reward:28.424\n",
      "mean reward:43.798\n",
      "mean reward:61.960\n",
      "mean reward:146.758\n",
      "mean reward:241.444\n",
      "mean reward:465.475\n",
      "mean reward:684.869\n",
      "mean reward:701.121\n"
     ]
    }
   ],
   "source": [
    "total_rewards = []\n",
    "for i in range(10000):\n",
    "    states, actions, rewards = generate_trajectory(env)\n",
    "    reward = train_one_episode(states, actions, rewards)\n",
    "    total_rewards.append(reward)\n",
    "    if i != 0 and i % 100 == 0:\n",
    "        mean_reward = np.mean(total_rewards[-100:-1])\n",
    "        print(\"mean reward:%.3f\" % (mean_reward))\n",
    "        if mean_reward > 700:\n",
    "            break\n",
    "env.close()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Let us record a video of trained agent**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_animation(env, save_dir):\n",
    "    try:\n",
    "        env = gym.wrappers.Monitor(\n",
    "            env, save_dir, video_callable=lambda id: True, force=True, mode='evaluation')\n",
    "    except gym.error.Error as e:\n",
    "        print(e)\n",
    "\n",
    "    if not os.path.exists(save_dir):\n",
    "        os.makedirs(save_dir)\n",
    "        \n",
    "    generate_trajectory(env)\n",
    "    \n",
    "def display_animation(filepath):\n",
    "    video = io.open(filepath, 'r+b').read()\n",
    "    encoded = base64.b64encode(video)\n",
    "    return HTML(data='''<video alt=\"test\" controls>\n",
    "                <source src=\"data:video/mp4;base64,{0}\" type=\"video/mp4\" />\n",
    "                 </video>'''.format(encoded.decode('ascii')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<video alt=\"test\" controls>\n",
       "                <source src=\"data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAAbvttZGF0AAACoQYF//+d3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3JlIDE2MSAtIEguMjY0L01QRUctNCBBVkMgY29kZWMgLSBDb3B5bGVmdCAyMDAzLTIwMjAgLSBodHRwOi8vd3d3LnZpZGVvbGFuLm9yZy94MjY0Lmh0bWwgLSBvcHRpb25zOiBjYWJhYz0xIHJlZj0zIGRlYmxvY2s9MTowOjAgYW5hbHlzZT0weDM6MHgxMTMgbWU9aGV4IHN1Ym1lPTcgcHN5PTEgcHN5X3JkPTEuMDA6MC4wMCBtaXhlZF9yZWY9MSBtZV9yYW5nZT0xNiBjaHJvbWFfbWU9MSB0cmVsbGlzPTEgOHg4ZGN0PTEgY3FtPTAgZGVhZHpvbmU9MjEsMTEgZmFzdF9wc2tpcD0xIGNocm9tYV9xcF9vZmZzZXQ9LTIgdGhyZWFkcz0xMiBsb29rYWhlYWRfdGhyZWFkcz0yIHNsaWNlZF90aHJlYWRzPTAgbnI9MCBkZWNpbWF0ZT0xIGludGVybGFjZWQ9MCBibHVyYXlfY29tcGF0PTAgY29uc3RyYWluZWRfaW50cmE9MCBiZnJhbWVzPTMgYl9weXJhbWlkPTIgYl9hZGFwdD0xIGJfYmlhcz0wIGRpcmVjdD0xIHdlaWdodGI9MSBvcGVuX2dvcD0wIHdlaWdodHA9MiBrZXlpbnQ9MjUwIGtleWludF9taW49MjUgc2NlbmVjdXQ9NDAgaW50cmFfcmVmcmVzaD0wIHJjX2xvb2thaGVhZD00MCByYz1jcmYgbWJ0cmVlPTEgY3JmPTIzLjAgcWNvbXA9MC42MCBxcG1pbj0wIHFwbWF4PTY5IHFwc3RlcD00IGlwX3JhdGlvPTEuNDAgYXE9MToxLjAwAIAAAAHoZYiEACf//vWxfApqyfOKDOgyLuGXJMmutiLibQDAAY82AAADAABEaliezICUal8AAAMAQkANQH0GUHmImKsVAleIOLiWowAfaD8VcfEgkGFreTAdLcypLdLLH44usPNhSQmn/BicX+HT7d1HJRBMbjmzdxqBK2Dxaaf+S3JOov02qh8GRvFTQE7QY0gBFMEh5E3lVlKZgowW+vT4xvCEA6dWYHgGxyOWklz28gU8cRh+auTyqt23o5z6n37lIrU/nUBuzxfpmkEE5LYaORDpm+g2wciugBhBg8GNLkG/IY1nxAdBWx7JDYdeQHSDoA8Z5JZxFIe6fjZyLe+qv/ccIxBB0J0LbBA+italuS1hWe1VlzDZNo8atENYBBJz0h8WRlUcLFm76bApb+AOcBX/M5+6rrXTRbCIhABKIejHEukOE/JcTpP2sYNjl3uvLvekpRb428v/c2NTpqtlPf2yEAdpyxDFONlQbuczJ6PkuEYxScBoipAYCLyKWLUay+iO5gchST8blh7AXkm1Wuo6iqAwuwcLfqXJe36dLTgW0M3qFcuN4e3WsA7QSVLBawgFBobBcTowtTYXm25GICvemPH9DmxLH4zvV178FXdCLOE+pX4n9wA9CAWm4AAxwAAAAwAAAwAAV0EAAACOQZokbEK//jhAAAENGl4L1qYALoKEjzj/QHeQgityW6LFewXOBx5c4M7Iac2+3mRNfL6JT03B6A1YMcL4LyHP0O/f7yeqO6UuMtusFtlBEJ1gJo+5kSVO2TTTuEuqiBoSYlhS8QdTNu0WtOyNQP9/KwqlLUGMfGJ9dhkIjfqJZ5DMyzO70ZLA9+Wnw4dgkAAAADVBnkJ4hH8AABa+6vAFaks1Q8dDviT2VB96qeiJCryoTv1fxgIn5GuNS1A3b3y9q70YbM+DgQAAACwBnmF0R/8AACTC8Exlv/tPeMaysaTfvgIsmMlnRgATkMDJmnoudjFVjAw24AAAACgBnmNqR/8AACSx66SH8iEOV7JBW2aZ3375P3jhUkMg278/foy+6DKhAAAAl0GaZkmoQWiZTBTwn/3xAAADAp8DG6AIPNLrZvQeuwEAWXU2ulhpjtWTXQGFGvhT+Zbm0qQfkm05K9DO58EAos59GFjKGKbiN8XptVzItbZ1G43tIn7/SaJHR4MYKdkCTPz+t0fbrvRTj6pJpRbvNuqqULhgixTG6oumSbmNamk0LEiw+dTchs3H+RE0AsSodLwzd0WfaKEAAAA+AZ6Fakf/AAAjvx+MMZvvT/l/C8OpgJsWtJuhYwRnrNqGW/XarsUtQSbR1nbSAAcM02G4UxYvM5ChCP0XYEEAAACmQZqKSeEKUmUwIR/94QAABDNbwHSmq9AGxwYwfJzVuRIW5NfqMfg6cqQM+Qj7bR3vnnmcuVrx7f+j5uUes2auOGh4pfOxDlRobUw61dTrlb8mE9A4Fh0g5+uPbXYKaUzX9NpHd/FGU1JLxK2W61G4TTw0zeiFzuzGWzvRRs32T4eJF0DunssZNo8VjVbCQDCf53bb3iYCLhsFjjcRuNIjvAdNNe3SgQAAAEdBnqhFNEwj/wAAF0w5Is89qu44flywfTXgSJ5HXerR6XxAblRoKwU1N+6ktYBBi/xLp7FQAhOEY6w9XchNtKLBNLKH10BRvgAAAEEBnsd0R/8AACPDRwhwPNWzWpNAIlqAtNhHxfnCjUJ2ABEBnIAIOqHGA0gZ+ZpjZrt5+44dKBd1m6Tyj3SGAlqxUAAAACkBnslqR/8AACS/CySVvBF4kfJC5Hyy+XYVq+lksi5MawuBdBhfN2o+YQAAAHxBms5JqEFomUwIR//94QAABDNljPAyQZ9t4rymP64sFUySwAvjO51sh7qk0Ys5a6WyeXjsvge3chjF5dbG1kyqA5StS1AwMn+iO4sAQhLGN5k/05/rfI0KR/l2a8kS93M/ecYEl/3900wOWBbUnQ8zfEwxOf0DhpP3i4WAAAAATUGe7EURLCP/AAAXS4pJmilhLY5WGIw3s9bUknFb7vbFrE1JvYgAhjB9icLYsza53fa4qrMgAWkDX8yZcLfa9g7rO38ktzGn5QFDpt4MAAAAMQGfC3RH/wAAJMLs4XHTtjoQ9YlcNp7KlBiGEIp9nApmSH5rsgYd2wqutlrozokZLYEAAAA2AZ8Nakf/AAAkvwQmbGWRxpDB0pjvFKJHhEWJ3ZoASmwmL/zAgdV/GmOcGYT0kLwqr5NW6TthAAAAdkGbEkmoQWyZTAhH//3hAAAENLlstmIANvvNi8ZzOxHC64dlyU9OHY+fqUbW7tivuwkfOrwiG4A3G7venh16x/U7FKxMX5R74tZOZxtVz9ckn1pdIGF32XffoY09iSrPmUbXJ8WM9KxZ/erv4xLwVS1OWaSSOiEAAABDQZ8wRRUsI/8AABdDSCngtmvAJI0A0aNNRhUMd3557qK4VWEk2Mm2OcEzp6wFXj3wv5qcaXpJW3ETMeudlnHyLw7eOAAAAD0Bn090R/8AACSr+FgKsa42w8mDH2JGvl9W/uWcux5xHLUrrvYm1cQvr063FfqOORwvrIb5p2sLJLPjYDtBAAAANAGfUWpH/wAAJL71PiDt6BPm5hEePLgM3jSt9ghoMaLR4RpD62RocyabYF2Vt8bFIZrsPCEAAACSQZtWSahBbJlMCEf//eEAAAQ2itLuAZTRa2plD1TGbHz4eKPFzcuJGDFSHFPda7/vQIi0TxhDeIsDRz0WSHUCjBwqL0q0A9j2QJtqqANdE31dqHNTPtNwZ4AzwiWQnoR5gezxHC0guU7Vg92LnV66iy4NmnkJhw5CtF4/C59OTbdAs85rzMOZTrm6oCe+JZfZJ4AAAABVQZ90RRUsI/8AABdLW3XIIvlPg/86R63MQZg2WUYF7jtGVYs657wzfEYpbpmRm/vQtMjAA2gJLDie0jcq8zq1LdffQlh/BA7buVzduJu/g4snMMIo6AAAAD0Bn5N0R/8AACTAmR/PZuuSIgWKU+O1PHoWUlK8mCCmceTOnjAyS062XsuRTQIpJsheAB44+ZHpF0QLqDbzAAAAMwGflWpH/wAAJL2DgcoHg4p3UkshOvMIE4j7vWz5VvmQAQAy00AP37q9Mty/3qv2HG9hgQAAAI1Bm5pJqEFsmUwIR//94QAABDS1ejoYGYSpNcybLXpp9BlY0QCmEQ6kNE4mfmwdfaGsER8Z37q0uAWK7v/cBkswx3wldOZL9J1uPiPB0+sbGks72s1dQPqd7tApzYXPqvbCXlQnYq6VtMfgadzfSjiG9+NSTu0OGsZ1Fx1cTff/ny1pcjrHV6h+j1KEJ8EAAABHQZ+4RRUsI/8AABb+TYuGkHX6/sULdMhfKgoaVAvxNmvFhlgTjviZ9UmymEqhhyCyxa4ILCFrE5yXmhmK+3nqQ8nGJa0Q4BEAAAA5AZ/XdEf/AAAkkrqxiozlsP7N+kZkutmDnXHKXvSmtHOv8ui5dxq/cxrHadG/mXJ+4G8eU6jY3NmAAAAAMwGf2WpH/wAAJL2DQmTn9EERjzmF5H8QOyW5s5XaKHiNZOUCacL8r2R2YvxEbgSj2gz2vQAAAI1Bm95JqEFsmUwIR//94QAABDS1fGKAHQGRgxb7l+6HO9KSDAU9MyM86wFNptcwoJliw4XwUqk80stzY28YQIqafeWcYYMb6u/mzngwrVirqNEbthaph6udWayu+EjVgHnSCE9boKtDO6Tem5dX2I4oeZLXrT0sPHZzLegdvd1LK7JrRNmaaCLHA8AHRrAAAAA6QZ/8RRUsI/8AABc9NPf6YHavdQzKCncY2Lv0OjDR3B+NiK484+A7dQDCZeX0ucND0XSZJxjM26vvcQAAADsBnht0R/8AACTDRxuqfH9tkNsfe6KF25u1ZXtvqvfhX2wrwkKA97w62gpXlJBBActZQHs3FrZh/0qtqQAAADEBnh1qR/8AACS9hB4R1w836NjPta+pvXnhiu8rG03WDa3S6cdhkQ9y/muq3BHql21IAAAAbkGaAkmoQWyZTAhH//3hAAAEVRSbRtZNEAG2sRnbGx6PaDyCtgDn+7rXj0rhMFVWCpirRIDUSIp90eC6DUBMktvD1Ammmwxz1XX8vgpKwxnDoavhU8C/pX6etQSo+9+x2MwmY2LUa8lXqn4Q3JlKAAAAUUGeIEUVLCP/AAAX9veATW/XBJvzdelAyTx+8Vvp6BtAeWz6bUxHn2j0LGMqoHOB8CPhYZPVGXOEUMMn10l6m12a0vwFWsuG4uaoIlVq9x97gQAAAC0Bnl90R/8AACXC4f4hWTBuiyYaIhF3I7Lisl+2DSTuzvhpJPdMzzeQc7/28e4AAAA5AZ5Bakf/AAAlrl1U4dfJuDFPL6hLm8qdV+kVi/Xbfy3JyV8LIaR3xT5BOKgATjQf3nzi1OJn4MixAAAAWkGaRkmoQWyZTAhH//3hAAAEVouz6oSH1KjWJOiS0Rlj18rUokKAZXNCevx4MRCkVK0KdS4AGtzZ6JffNHJVg+5qOmDkOyS5Q/c1xTO4FAtNNUAxWViFTrDHrgAAAElBnmRFFSwj/wAAF+ibYR9ddTRCS3fbFopeCx4Gv5wF60jVPJkeUI80dJRAC3Bxc5CmOd9NNTv+o/ZjDNQ2g7a1acoAVFuLShFhAAAANwGeg3RH/wAAJcCZH89lfIDtPcxrmoxYM5eX10tRL9hHymnB6FomohTeby7/dd++DPopmdUc4sEAAAA7AZ6Fakf/AAAlrl81cvQjztyB1TsvjFyH1THicbyZfOBCmgDK/otz0AAF0N6OnupAUf4zHzpX+3X0vusAAACIQZqKSahBbJlMCEf//eEAAARWPz0fhsAL3FM9mVgirxo+C48FoNb+9xPBvK8loFnXZaTCzLm6RD4/s9SVy/eo9Z0iKK8zHV++WUSkJKWeDKGTRwZ/pyH9gNBAlgrGXc9Az0AM2R4/Ck+DwSxeb3XJdLf2rBhLhAAujd7SH0WXG7c/i5F5mixjQQAAAEpBnqhFFSwj/wAAGImFixPQzwfiaRLVRa8AAN0JMPYJBUOLrEY0hmpdp44qsFttnbSECClGKIFDnvxklowCN4eQtP4udzQFDBLbPgAAADoBnsd0R/8AACWp6dtQbizEhGZGukdNqu1DTVAtf4PV2KgEbz7cGBzeGGw32ssSlBwsyUFryKYeZs+AAAAARgGeyWpH/wAAJr8K6liEp4xkqFwNIsyhoASWzEh8lYEi1YyhfyBLzq4or2IaeLys/Rb6/UIsugNVkW5LbmQk4kkmVUKh7TkAAAB5QZrMSahBbJlMFEwj//3hAAAEe6Jrm1n3AMABMuG/aQpIQTLgVcoF3FonJHKM7/FLk512fITwbdbEHKftF86wjQ4+bHzD3+6fgCwmXHY1CJs0sgsF5v50CS/T/8MZW0QxiM3ZMviXAl7t7hgm9//LI1Ms9Qcw81sGqAAAADgBnutqR/8AACax/P+0psfcrZ8vG+OW5P0rx2ZIUaeMzosDsCS2wF2gbcNtmjjMdvWKn/eJwp7Z8AAAAGlBmvBJ4QpSZTAhH/3hAAAEdxSuq/g6LDFyqPyI1mq27wPpVb3UWX8Ly50nhWnCAn6prWquNyZiCHpB1O2W0pyHyR/MbJJeEkH3ssCzYpYTyBlBJ+S8ZP7WALaGHBU7jgibjMOyHHE+M+EAAABZQZ8ORTRMI/8AABiLpS8C+IkIB47EsiHnAACd1I6EbwBqpJvt0jKkOLoZShtsafKF0q2f2DVqYwIvrk4LGNmr2Ddr0A0iWNRU9X/v4SnGEA+UQRKZwiPVaBEAAABCAZ8tdEf/AAAmqelbDNbu+1KACMixf0zAdeo84SG4GeRTSkQoG+RuyGyu2hF0QwmyDIa3nBvMFNVGxXxsALBf4rz5AAAAMAGfL2pH/wAAJr0mv1SEdL933Oe1AHqbP/27ulx8TZAz33h6tsnlhuszidQBJW7KgAAAAH9BmzRJqEFomUwIR//94QAABHUU4+RIBBxF0X2tvZLztuVYL1YTDNFwEJ52YuQvVg+WQb/8Npqz7TFdnVFRtdnnwhmG08T0uVZPb3U/mjeZ/9bkpgU0+v+yCIId9NM7GegA8OPLSXBg9LKpFHOf2W0zbNI78VsqjjHd8PPFlIRIAAAAWEGfUkURLCP/AAAYeC0iIVftVKYAHJvGSRoP8PmJs4OXY/Npd5XeaYlC251hT65UuZ+j7LyVl9JbWDnsmpEK+npkKk6zpPH0ZOwEyBcgdWPV7urq+qU5ln0AAAA6AZ9xdEf/AAAmnzmK9/z/Zt4sLoV1FlWMz2hAwU0A0EoshHQ4R9o6pj6G3JF6r5+xDbYYOt0oC1DnwAAAADUBn3NqR/8AACa9JwHmv6I+77VmBVNo28UN98COBauRl8AAFzhk1rHnWrltCLhYdAtrlYdwgAAAAH1Bm3hJqEFsmUwIR//94QAABJNQNXsYiZLPBsAA7r/LkFGep5BL2BI5L4Py/1N2iQ8wZMGvoBxg2UAFjd98cIMURQtdFoU9amix6m/0R+BLwpnEy1rtKFlUqb/rNUAOJVQP6IAdmyN8nH8d9BnBluVS+f/9i+eHQVakp4dbxQAAAGFBn5ZFFSwj/wAAGR91NRaPuTznmFv2aocljxGD55KjdW/7biFo4NLFtpHT5yYC1O5SfwqzOdmj6CTj2ivziOMDm2vYn8du2kfq3opBP5FDmSDMkZYWODuj5TeXJx9HWTygAAAANAGftXRH/wAAJ940DlhXg9dbgRu4/OTIrrBcl8yLCFAJ1FxUgXNLLHTWmN143K3Gjvq/ZUEAAABCAZ+3akf/AAAnxBkycPp9L6vLyHeiU9RGtc+yxg38XXk8pZHp5VZ38o/brACaYaHpp7LSGMeJPMlpPx3PK3KCjGBxAAAAckGbvEmoQWyZTAhH//3hAAAEk1A46GBn/ZhzwHwSHQdC7YP2081o7QRG1DMJBoGdND1EkSIi+j+RGaAeTIXrxXeybMwI3tGc4uIuXz+54vkiAPTtQ1GcuIG/tntB1XviyJ1BhRfv9e8aIJW8WZCp0UgVUgAAADtBn9pFFSwj/wAAGRAc8tL7f0IFfMAA4zoug/oFT2Y9XH7g3pcyeh9ENbExdJV6N85DWSRc3osjcMzYsQAAAC4Bn/l0R/8AACe/npPHX341AxHT2VtpKRrWtq7rAMsNesftBvFFlWPW3orVJsWAAAAANQGf+2pH/wAAJ8QZvJnvkiuMC16ktu+PbwxkHegJD0e5qgX6qyFvZyACIOgz2UcAbjuptR1hAAAAc0Gb4EmoQWyZTAhP//3xAAADAvsZkFKABt5giJT3DYxLo8dhaP/BoZxq+gsj6bHC6nLahBImL5MNUufM/nTr80ySdhzVAcb9+mSACyRGPhI56YkiX+CF6xThDJMDG2VAI28dybINuUH5uGRXoWejDkwzGkEAAAA5QZ4eRRUsI/8AABnJhs4dL5l2+IST/q3KyOdWlPClu9+jMAQUF7peCbALaOvUCE/QdjBBqDh7OftOAAAAPAGePXRH/wAAJ9qd/WzsjFBh+m/Hc0qUWIt66U57xSCXs6QAmroU+KipkfzxnshT/0Xla5Yo1l0R5CmusAAAAEYBnj9qR/8AACj3M0VUgubK4SD67dCy32p4HPSxKbnxLHEO10MYTAUiYmLY7D3httQsNP6MAD+dkOKWAI0jW+l8ajDPWbdtAAAAc0GaJEmoQWyZTAhH//3hAAAEtRT5SWcp0A/8qP9AA4NjhNU2j0NgfS+Xax4MET+8Eg6M9GZs9fusxTVgeSCr6fCPMqm5cQ94703uoIJSblKhtIf/ZzyYnIM9cMReFW683VD6aks+val9HVCXg5EjS3F3m1gAAABHQZ5CRRUsI/8AABm/dR0LYZ6SiJLPC2cZtP7cFYNsuwDdhn4fARBpji6grzOAX4U4AWtZXwHYVoOL0KkIZOn7gng+f2AHZYEAAAAsAZ5hdEf/AAAo+YEMJKVsmrsk++bq2SsX/mj1sSExJ9D7lNbd/OIVR1Uc3ZYAAAA0AZ5jakf/AAAo5Bm8jawrR3VyouyhFASR3gt/8G1TmgfGlgpGlld+KgLcjWVOW6u5Jz1gQQAAAJhBmmdJqEFsmUwIR//94QAABNNODL3/2GALWbMKMFgHerfz2L5cwld4LMUfFcsw5h51P7by30/YstIfZeTKLkFn9amvexyteMnVDOT7mF0l2A0HKC/OV/Dx63FPhWNNWtiae8leP3qpV23yiykUl6Do7hEHSyuYK7+k7+zmQU1bXps6MyQfIOA0jv+lA16MtcMvjezfCRlzpwAAAD5BnoVFFSwj/wAAGmmC7bAEKPWk8AAU7zS/DmdKRe0I6P6g2nahAHUQ9/56q6hYivCuAECt90wQoyzCLrPIZQAAAD4BnqZqR/8AACoZgZFKqG4/hH3YTlc7enYxO/4kdKVEnN3Lb8rWEALCNc1MAH3xCYYESipdNvkZJHxxSMs7hQAAAJBBmqtJqEFsmUwIR//94QAABNNTKsgAHRC+srLjWV5Vmu6GXD2dxS1uE0UKvd6fJJkNQE9D0NreYRPaJa2mqTYNu5U6BziR1PaPuO7Nknw1RnARaOZ+WcOYJgw58Hz+S6/J4XGiAv58/a19Fd2tFIPR9G90qJg5HSZSdLoYCNahvpR0ry2qPVW+CxObGWk/gUgAAABAQZ7JRRUsI/8AABpodFrRp0KNcKMALbq8jFMjUhreX+3qxnjqIBcmNGpLFNqS/3MJ+VOtfUZL/fg+5GhtXJQHCAAAADsBnuh0R/8AACoanY1iJ4KU8BGvdHWXD4+BZQCB3V/Sl8gTRnirkoTwAD3rq1oDSRm+BxDvvp1MUc8ypwAAAD4BnupqR/8AACnze9MEMsJv9aP/kQKX8MQzbRqWlkQZBozmijKmVh2Am27c2aQ9wPdG3Wh5Jl9xPFPUapuvwAAAAGVBmu9JqEFsmUwIR//94QAABPWR34oBR/ohfv/9t7Jf6moau1WyynQ2JnhrP1oEH2y+OdnaE8raU11SJzl/AM0G3PsebfWOsIxzVW0Y2mK5ixgmzAX1P25H0vCcmO0+jn0sztTN2gAAADdBnw1FFSwj/wAAGwmFi/1TwAiJF8FYIScRXi/Vxjh3yP6Oqi8W4X9u2m7YwLivWUAP0f56fLuBAAAALgGfLHRH/wAAKf+fM6VQFo55ji500lwzio606ztXlSeFe5PIsm4oPmSK0hxLrJEAAAA6AZ8uakf/AAArOVy3l13sr2rxcMKdhNT0Djxd8qWNGgASxT23nbQe6ED1ZrTcGof6TBaeiXtZh4YaoQAAAElBmzJJqEFsmUwIR//94QAABPstEuto22oK5IJEAFAhc9TcrNcbiiYYUrGUWY9fr1AE91+3Kge4lxGDsgKusTjPRmTlZSnRBoeAAAAAKUGfUEUVLCP/AAAbDXybLteg/1k9OoGknjRv4U91lkqJmcSHIcEeDD/AAAAAIwGfcWpH/wAAKzcy4ayDyRhyGv642KZYhiTG8R910T/Fd1/BAAAAhUGbdkmoQWyZTAhH//3hAAAFGZNYEfgGZzQ3DhA/ZBFjw55kCyyQUJhynxfR6ddZ+Nayxq8fHNJgd3gtBJP2Fs47cbtIfmJj8b1Rs5vRPNbFQS7yMnvdfYAgZ4IHRNtH65UPVtIWKT9UzvQTOimkjRa94BjSzs21iAifHdKsEnmDhv59cEoAAABIQZ+URRUsI/8AABupfHO1kgBYbu4wC/2nj7uqQ8P7FnvFTtrkt8JXbVsmteiKsRfi9jaxt5wqqrWbL4XcOwbIo14Frr4IGaCAAAAAOgGfs3RH/wAALF4UW4Cw8KcKcvcs9oQAS1G1OEOx1M9A3df4q3CXgzhhr5kLNwzJzZ/JSvZ6zp0GLBEAAAAsAZ+1akf/AAAsY/OS7cN6pnzmsDMIaiE782dzicCPECVvCv6gT+Ua0OX3i/AAAACjQZu5SahBbJlMCEf//eEAAAUdpmKLgEKec/dd8nPcoXT2Z8Pva/Gl2UHOD3/aElZBh4enTq5UktZn2GH4kA6RPMKsrmAVYRiBvjrKuovzcmGQfdTehC3FvRpmtEDwdc27oTJTnlz5bL302yIIB7Ic0Hw8Tn8umQNfAtX7z32hf7+KJiEGiyWzbyFFnG735cvAaMz/Ckims44eVdhCPrOzRgk5QQAAAFpBn9dFFSwj/wAAG6umNacVtMTCtkJr2XoAFraNHnpsqcOxsPtaytOmQyYjbCVgm1yn/vZY+B23lmNJsMN0WYRed5k7CYTCOTwN7BLkqiZPfen4DUZzf+ahrKEAAABEAZ/4akf/AAAsRBkyb6WJbqxJ9ZpCgjQXd+QuwHCoAJHW6zNNVruky6nNNmP5D6wNKMfVJJc7JKUxKHXoR+6Tmwh1eYAAAABwQZv9SahBbJlMCEf//eEAAAU9k0n8c9+CwuU3rcNNMrUOKPSNAC0Rti9yibnRbhe535c+pXvonz9ZchqGnAMsSDqSf+6bk8SDeszNI57gR1KEjb+UIg2J1bA7DUn234NQ20hjEk5bRMBFFpFFdEAQgQAAAFhBnhtFFSwj/wAAHFh605FqwqhoR8FLwLlIBanaKUUALF3RFfFALQnKzJ6ZLsjbcuXLoH0K3xXcS4+GgMNevXyms596kFpybd0KFgCIVc20vQzdUOC1SiHmAAAAPQGeOnRH/wAALFqdKDWiIj/Ka6CeYh50/Zc8zUOjSLVnZ9wVIlgnfOYAA/YfLrYnSfEfELuHEDLHm+6Nm4EAAAAxAZ48akf/AAAteYGRSqh3iEzT+e47olCypLe2YN8ZkNNQ0KlbjycT5lCvHRkYie46hQAAAGNBmiFJqEFsmUwIR//94QAABUFScFezhZ9AFtGnkKnlvAKeHHgOwZV/UBIarODKaW2OkJShBnxA6WBO4atkQVg3nLIwChg5L1SDE3+RBR3SM3CyP/Qo5DpvCLjvc1Hthv23z/kAAABeQZ5fRRUsI/8AABxW93LzqDfyQuBGPhzUcgDqjiNPs8wpU9IWtMTp2f3MM5Jnb0wRRibjopNKegitgPkTFW5t0qnVKP+3JvTUpb/o8hk8ezdpplxAjhXlrZOtI9xWwAAAAEIBnn50R/8AAC16nY1hs7UBzZfuvQyrXuPUFJ3ao/IR6Q0rBDsNuYjS5gUeMAEtDT2EwjYfZT1Ne4Z6Ej1+GxcXfUMAAAAwAZ5gakf/AAAtdzMXsMviZtg4oFHXH6l5ltZzSZ6S3+YzhCQoWUi+Axu2WVJXlP9bAAAAeUGaZEmoQWyZTAhH//3hAAAFQVCJRYAOiwrgkSt9CALY17Ifx1IOPvNRipJNsnL6DRlpbOZtl0dbV/3STwxMmngYDHoukEGhsmkEvddXKuXkoaBurg1Bsf7TM263BUJ3SZTGfGmJx59/uqapOzf5KHFrgxm8Vs4RdNEAAAA4QZ6CRRUsI/8AABxXL6ag/1epiPbmc5Or1VEx5u+xo2Mo5q0O1HEnzNEg6w5YNNnFDMzO3VQBCtgAAAAtAZ6jakf/AAAtdzOFOQ6GS9USYyB5ihvtppgj0BeYM0kJuqyhPX+zhQdqzbVNAAAAe0Gap0moQWyZTAhH//3hAAAFYY0FigFHGwdYOkxr/FwTL4KnDWVrIm+bHHPdhzqdktps7nPVe5av2PI9SB/4H5N32NcWkT+swSXcCXZWgCPP9FEiA+JwqxH3N/oAXtbzcmGin9B6y6bZ03vUy1NHa3raxJQSsEsXhQr+FQAAAEVBnsVFFSwj/wAAHQcvReooOYMX6jF2uRjTaveBO4NLSgyzf++06Cq7OXe8tL90XQAgLUyXIvfpxiS2jhqeHDmOTeVxCt0AAAA5AZ7makf/AAAulzNFVILmycE7koPLdct0J5LjRlYVX4ECC2AAgSaGqsq8ZrPEUDmRF1a8vI0bVOqZAAAAiEGa60moQWyZTAhP//3xAAADA18ZevV0AUG0IGGe2qnUBJ/p8tApowfRW2qIrXTLRrCi9Z+PKt2PsAgUJQZEG2IHq9o5ArDnKUVe8svfsDyv2ufhgEC58n1T7NX3//6q/auJPWdcEibK8gLOXYbyOP1Db0s6kAv9By1B5JNLOtXSQwdbvLU8Pj4AAABOQZ8JRRUsI/8AABz0xLEalFOfPCijt8f7yEl2MEkELKwcTobQWvR2u+rYlKOiilhaA57LqbPQwLqABL5bHwsltlhM6sl6NoSUZmOWQFWAAAAAOwGfKHRH/wAALn+eku7wmpcwMPdfheExeJOmlP3N6E0kQYiDwRLct1Hgx8tSWuFjqgLVLAGyqoGzKrphAAAASQGfKmpH/wAALnjcBqbG0bKoJmkys8nz/5i4v2Cogj7YqRTnAwhVi+fzEPX9R9sQPSQgePkz0bhQAtWEP6QLWKk6kXSDIA3A1JAAAACMQZsvSahBbJlMCP/8hAAAFYSv+oFu533zj2a92AAv5p+ZvRT4iKRBVRQcEVbCGYBp0Ro5EblvYwsjaWFDV+O8Y38SO8CsR5uliEWIFlQgEDwgARWTT4KhOB78KDxv4kcGDF5HKKW1X/QEaDPvo4Efm3T9B4JcMt+0sRy6SHkN118nHKHA9FpiVlGLnGAAAABPQZ9NRRUsI/8AAB24etORbDcLtR0lfpgdZDkYGnG/bFPQwGYKzeY7tJmrro6DZuIqiYhZKIqttgRzmsubsACcbGXYNy1Sl2lt7/MNHPh0IQAAADoBn2x0R/8AAC5/nqC6VGVxAKG9ByIaaaslxtRtK+w3H3Z2A/OK1HzCfIw6+Gz/bXK8UfJOYjkjbakhAAAAUQGfbmpH/wAAL7jdRECJSAotkIxkgS/tmHaNWMPQE7fDkOxNyrNuZjztemfeuYZo7OfZED4AJpdtKNTEDp62NkvvMSvOhqNjfHEQ6fEayL5LwQAAAHxBm3BJqEFsmUwIT//98QAAAwNyVtq491zw79eN8iL5Bu4/QWAPihBomBbBHtkE2lsaYGCKKZW1TOG6IK7+y3u/98MgFrZm8rpDBfRuzpjzrPhkf0bgAt2s+QHE/QEqt4Z7RN/nJYptXa0cX+6YJED4Cl1+4a3hopmC8H+AAAAAd0GblEnhClJlMCEf/eEAAAWFjhDoYGYlSJW5UM8TbmMCK/Gq6sfae7kQ8nbQ0jjl8T1Zs7+pNetWXFePhtB/f7+ANhq5PjceM9tYi2BqOWZKIu+WW3jLlp9hZQ+w0ieit+zEJFUSM8P8ilUlgV6WdfLsNjdaEguAAAAAWEGfskU0TCP/AAAdm5gtQln1Y/5AM0nATB0UadUKqZnJjdxDJDrmfXUX86T+Gt6TnfO+NQQI6UdhkzM3Vj4rYxBv/b5Gv+95H7iC8xWGcqfL/wijB8WHQYEAAABDAZ/RdEf/AAAv0vGROYoRiOKTZhax/j0GXXsdZvHF2gOGQIVnwckii08eJCZUPMJPxXjxaHwxlt4KbbIwbWtoh06ggAAAAEEBn9NqR/8AAC/PqFMS+vqQtGPnFy8wE9gPd1EGXLID77/mQTAGh2HRjd6/N7gKa5fRgf+z1HPfOpfiJM8qeRgugwAAAHJBm9hJqEFomUwIR//94QAABasw5Q6oDGJawXQGuN4r4yB3dQykNIwibIfNS57kZWGCKTfUkpd+S3nt6Ryl8OibkGmRPErPTxIdijbeW/3o+6KvKJwittsajWP3gPP4nI9dcHzDcxDonwaZLLUn7FUK0cEAAABBQZ/2RREsI/8AAB5obJ3Bv6E2CaRQEeyEys8xqfO0Tn3nPuRL6mSXg3oW/Zp/M1tm+aK+cW4pyg0yt6odRzjPUEAAAAAtAZ4VdEf/AAAv0scJb/EUuF1BwGixExPTPfk1fxLmSsTq7o6U6xN4f7fPqLdBAAAAOgGeF2pH/wAAMRIoDO113TICzupx5mSUw7f1FrjCz5hySDGtEI301Q6lOgqoABNHad3k6aj+qe6f1BEAAAB5QZoaSahBbJlMFEwj//3hAAAFqYAubwH8eQBx7gtfv1AdZIG5l39Eull4mHSXLHUbKzr8yD1TuGEUxERxI8VJwTn0uGlxm2wxJrjxPw79kHD0G3JtZcUqGKF35Tpt41baF0ARo5lY3L+4l8+brKfVsMYjvr1Py7AVQAAAADQBnjlqR/8AADEQC4yVlEbxjIT19EZMVRU79zIN7q17iS4EzLbyrlfsj81ox8z42x1czi6DAAAAWEGaPEnhClJlMFLCP/3hAAAFraPHjvD+PWWCPigAsyH09WEvx/tQ+V+A7McjRTkMOAA9pVMooqDnLpOFBx6YjbUwi1VQJ4dE3XuGgFhWiVIlSd4Bhf9U14AAAAA6AZ5bakf/AAAwcVn1D1UgBEKmdp02fctA7KNDKdQcdAR8mvH6frFxXO24I5KW2NAgfJRBN4vUyO95gQAAAGhBmkBJ4Q6JlMCE//3xAAADA4hIOXtwCa5r5hC6FeYAAMzAvw3+g45fREyA6DLE1f6fsdEtMGwSJTcD6fycJV4u0DZ1WZ69O1RlMcjNnw84TQoy15HtW6Up4u4ISWAy+f9vMkWZZ3AXKQAAAEVBnn5FFTwj/wAAHmcvc6J2P6PbxaLMvRiZG9zwqMg2lSACVGZL8n77tG3DwZDULWhtLtmJ/ifxvq3RUPs0WtCnyWu+NegAAAA0AZ6ddEf/AAAyV20mN/n4dJwIMQx1EcT8QjB9Zcn9caicPfOoZlDVkmsoESJ2vL8TQOwjwAAAADEBnp9qR/8AADJeCEVA7PBpNKAEl6dPIXytB6FMGq5hKc81kj1++XTQbdrTjihc9mI1AAAAV0GahEmoQWiZTAhP//3xAAADA51ogTA8bVj1rbR5GFd+WgCP2Y4b1RdNuvl++xw8eQhRqMzh/Kq2nT6KZ4PbBaDY1CXdJAGEJBwW8/I+fPQNbWoi2ChkTAAAADVBnqJFESwj/wAAHxtoMNgaq2iVuRkXYo8MLxVq8XOBk8W7J0Wvy1U9w9O9oPQ2LOlI1Qpr0QAAADIBnsF0R/8AADI3/riXusWSF5uwYgQq0DZqvIdDQEFyTyRMrrkSQxh5OfDx0iBFQAEabgAAAC4BnsNqR/8AADJSPjiYkcYLxxxKmqK1N0J2mzFtHYVTahUdWAQA68n4uUMRBmvRAAAAbUGayEmoQWyZTAhP//3xAAADA51of7nrb8BJ4+9099p8RZHdSxWfRFMpUVBXlN+OYVv8sACKUNGB4qY4p07ZCcMozw/i/3/0NezYkJ2VXKpaD0Acjk/XI+A06j1oHVJVNFng8oTgSc4tbKqL1VcAAABMQZ7mRRUsI/8AAB8XPmdACw3drw2CP9GJBfUSpx9Xwd3q5mowZRg5cPkKmQPU7i2rudQs7/gP4Jtg6a+cbpoX0dxyeMuZas2K7X/4jQAAADwBnwV0R/8AADJTsfglhjueuabiQbd45ISpyEUVV8/ESlfxAOiibUAxZX1FSra4sZ7NJYIZaX1EnrvGr0EAAAAqAZ8Hakf/AAAyUDXEcSWQqdCxr5Kp513fyNIgAFzdBDlzoK4JfWJ6cS0wAAAAl0GbDEmoQWyZTAhP//3xAAADA59nffMjLWQHEV+LJcM9G6mDBLr3OtTjHQQtUw1gqIc8Vvly5ziazHh0VQGzpL6GQJYEBGwfHvK7YOT4JsvoxKjvEzsDGvz2mQL+OUNhsgtHYxWSdl1EZKZYqkRZ4ZRkz9N/Q5RrpaAOV+3y86CspzqMv1CKaoGwg+SMsHjY5fzFDskbdhIAAAA6QZ8qRRUsI/8AAB8dTdiCLW2GB/AaJH5g+Vux9yE0N14yLZpMVqZLCCFEYxbXQVKZWoZdAj5zPtlvIQAAADEBn0l0R/8AADI1Sv5WpT1CHMf1wlUALEt99slnrhHWtCGPzAz0scxxLTEgHJyB94TcAAAAMgGfS2pH/wAAMjpQuE/oGtUKJz5bpSTTY/jsxkSvqpXL0iDfFf9QNCtLFp3c5OmS9ebgAAAAVkGbUEmoQWyZTAhH//3hAAAFzX+2sAoJIAgH45zKMq7AW1bzAoK55b/bAKIpew4RRYvQe+xTqVRHH69EEKmQ2H9PY9Mag0ucZtp4sCa8+o1n5cDLPA55AAAAW0GfbkUVLCP/AAAfyH+B0JcUYvZoVubZbYI5dRjTPCAhJrtYRlfeQEi9Mj4ZJIMEQNxL7pQ2k/l74jg/6/bMwGl9NlLavjg/fTgQbSLXKWTsxo/aQvG3VGs5iDEAAAAwAZ+NdEf/AAAyNUw6NG21KBzX7imjqmZKxu1TERYLNQCwXEMucqqFok+UfSw4qzS9AAAALwGfj2pH/wAAM5I+OOE0IyVthtkp0edYA4Zh7ZjNcDKNveLjQeXgBJYxJ2f5n/AuAAAAc0GbkkmoQWyZTBRMJ//98QAAAwOeRUlgZf/zFdohLHHoxdRQJDq6CWEFx7VbzBZH37+kUuOvZwGwHYTJ1gLxsDVj+tKxAI9CyrqniJPRI1BbQ/NSZ+7Aex1lgIJmd04ffxWeukzhno3op1e6ZFxF5ax1FUAAAAA7AZ+xakf/AAAznjjNJ0FiVfvoaKAC0/t9H0X9UB5S9w4G4cNzhBreZ4FryyIQCo5jYgs0Sksb7eWukQcAAABuQZu2SeEKUmUwIR/94QAABc8tRtQc5vMRjo3wkT1u8+Iu2LHE+GkWtvjs+7EWiQjq2JuulBArOReae0DYQubmEy3OKu+XGz3k7u1yMg6GjBqqlhlStH1wQuzASZgHKtOCGypY8jJwPFbJsdzGgcAAAABJQZ/URTRMI/8AAB8XdlccELUv1Nd0+vEimtolLblsPCFpumQQWsAsBv5rzo066U3jVNrw/7aRW6aV9Lu27Puq6lOU3V5G2wg6XgAAADwBn/N0R/8AADI4S4RQAAJY82/vOqaBGpnYqnrhzlQ3m4vwUaRQjq3JVohMdRIvXQqScTAvbI8SRTLTpeEAAAA0AZ/1akf/AAAyUDgcYolQ9TQAkusjLhyA1hJnGGQw2wj73KHYMsxDVQhCXYYkLxCnlr1ZgAAAAGhBm/pJqEFomUwIT//98QAAAwOzdLjdet10PdAACM2quT5n6h5Y+ZedSe2H89dfLflAEQSglOhw343K86LOSQwT8VgZSZ4aEYlt6jQ7iQYwDUThUhwIiX+eB6DU+Ce+cAswSwVAHtVvRQAAAE9BnhhFESwj/wAAH720tw3hUxEjbR1agBHbEkmcjPjUZndZmoy9BdHESlZPUN7dUso2DEABB4c/VXQhGO4CCi2JqSKrr2SsYTL0/wcEfpeBAAAAOwGeN3RH/wAAM5c05MuDtn4Xy/2VpAAA3XQp+uNrCSpCwwUTfF+h0J2rFp4Fg5fjAQ4ko+0kL/HiT5rMAAAAPwGeOWpH/wAAM5JUtDDNL0NsaMavkgALqD3/5+NIaqNI8+XqxW5PdY9wW3FPJ6Gzsvbbmv9o1uKYi7YSSl/84wAAAF9Bmj5JqEFsmUwIR//94QAABfRbupgw6k5EImdvfzxZ3gEyiLTyEyqkwJAKiUmtXiNuK3CmMLTluu6Mpe1g9XLAbM65hmW1TJ/sAU5Xn5jR47Bh6xQeB4slEt+SXuJnwAAAADNBnlxFFSwj/wAAH720tw3hU3O+AArBt/Odbh18HfQ2W68krPczYcakrcMqZVcBJdHQqzEAAAA3AZ57dEf/AAAzr5MARLLKmrtBYPR38xwKXAVgmclpy72A83e+/6LGQonRLcWmvk10MSUfY7g5xQAAADgBnn1qR/8AADN+2h9o4+gokzGgpg4Y6byy5ExMyFJ2o8eP2cz5ACZNCTg52m+7fwzTGD4xtl8N6AAAAHBBmmJJqEFsmUwIT//98QAAAwOzdP7gCFWhCEwFlmRlvAHEA3kZsTkEYl65b1oZzKpdnDEBmZTknF83qUEMua8Otsc/Pv/zGLl57UOLQYJuG9HyrMq8oXa81MW4gk/9BXPrdaHlm3GpuwHW7frQyPbQAAAAVEGegEUVLCP/AAAfvbS3DbkcR+gR0hl11QxRmL8rP7i9GDRjjMk2+fZfveag2eSb5JSV8woI3ytzXtYNPY2FXFOS8qYKGB3QO/IN/nsptdedisn0vQAAADgBnr90R/8AADN4A1QRn7lm/DXzOi6CNa/U6Y7mjcDv+e2bbEUwAAJq3W47YqPIzJybgFqidGBrMAAAADABnqFqR/8AADOddcoe9R42zGl1gi/8p6o3Q6Z35FV0mb9B8KsDPuIAITKe9J2enzEAAABkQZqmSahBbJlMCEf//eEAAAX4hinaAyBK9rr6vRX9FFNKvMZ1ycgp7bEENN3lYCzZcMwSoA8DFMtgTyFAu9kFezVk5CJ0krMrze5U7j9GJjqAQ24U4+jFFtjALDg3a0L4WqNm1AAAADRBnsRFFSwj/wAAH720+3FgpTAl6qDuBC5wxlUME9dVzCJzghpZx+hYBaM3twVU4aEZtNZhAAAAIwGe43RH/wAAE1X8L6q5lAztIszNIK6wmI+EDVJOZZ38WCxxAAAAIgGe5WpH/wAAE1jmFxVdfMA9xL/i+5uObOfPl9Sr2+TIAuEAAABgQZrpSahBbJlMCE///fEAAAMDtsssy0uw8xoemjmug/2Cz6HqXSq1aP5SXnNVzigDec5hc8GTUtARo8zPy36v/aHadobFTCJLlUxm7Nnnux7xpQdMQtdSJtiTuaxVhcyfAAAANkGfB0UVLCP/AAAfyIx+VtuoRaAE1dRTNxxn4oJ73PeSmETdNxS1fbbZQaps6VJyI4w6E81QYAAAACgBnyhqR/8AADOSdh2L/EvY0nQZCFkhIvLEJDNRjUM4mMG4NSz+DCHAAAAAcUGbLUmoQWyZTAhP//3xAAADA7ODksAE55cKGTM9qkg4g/6yiwXLv5ufY1YM80OTuciIeypIt42r8u6yQpEy0GR2j5xMS+Y5pl/8RPfATYSGaOWjj1X8p0/0aT6OmjOmZgf15JF+GkiUwnyvbEJP3UeBAAAAVUGfS0UVLCP/AAAfsIL6tDr/skeHU2ABKinVKsDFykLf4ZMfr1T7P4GsLO663pJZwDLw/9YGfPVNOMKTbYM6bnzXkXDzAih4UlYNm3Ffy4gqlDlHkvAAAAA9AZ9qdEf/AAAzl3S/3L5Nk3uyhjWLABg9R0ieJKAmsmF7uvBcEw/8K/+099yJZL6VoEb6SS93HeUC5M3KXAAAACsBn2xqR/8AADN/W4tN1dxWgZCqr9HeRNaEFpviACZhZmwzfYQ6+sjDReMfAAAAVUGbcUmoQWyZTAhX//44QAABiKT2RQNNJZfNYkgAbFV0dG6JfKld9v6qVL/M4262W6+LLxz1T0LWXr6Nrv9H3E3RNzxp3zLNNRaVwRIJS6U9+Ih+S8EAAAAwQZ+PRRUsI/8AAB+/6TSU0I/wDMLMr93MQOzuMKC5Q6wTWFMmCgWatJI1s9laiKXBAAAAOQGfrnRH/wAAM5d0sOjhoALeAI+9t5vgvQycE2YT0+DS179lxB6poaPT79FoUUYzVy/wqEYqfXwO3QAAAB8Bn7BqR/8AABNfjQstVvu/4w4AE4Z7aqMuzMoYQHjAAAAAX0GbtUmoQWyZTAhP//3xAAADA7YlilF0AMAQklPtfjFxptdocF+isNmbY+awunIq0H1eJxBQQTHOQoCzPngCVCgTY+XcQty9jETyNv4h5BpvbIG7Don81J0SS7kIIasTAAAALkGf00UVLCP/AAAfZBhUPrkHq7Q+aCNLABsPVZW2wmHV6cZsnbpC89ZNeHHe0QYAAAAbAZ/ydEf/AAAzeEvNfalcqLLxDRKiBzxTWLEGAAAAFwGf9GpH/wAAM39faqExopjGlD+c3VmBAAAAUEGb+UmoQWyZTAhP//3xAAADA7ODXuAmTEwCeq93VE8MEbevlrVByPqNJsMTtUig43xXx5ZmCyvheOY4GetAgoFnb1iqVhHz4t7ox6/lk+LgAAAALEGeF0UVLCP/AAAfwAIFRiUqpd24QAJ26lViOEkpNyJLkfxh++z2etM3XNZhAAAAHwGeNnRH/wAAM3hLzJZ+tFABcMITHfdD2u76PK3IB/kAAAAaAZ44akf/AAAyUnifhkODh9wZR510KJK+pggAAABFQZo9SahBbJlMCFf//jhAAAGHRSfYAoBVHYaASn0PQ2S18H3Y6t1a/tHtoMsZ58cLzBmDMNe0JeBQKNjr+sFqlD4Fb9q/AAAAKkGeW0UVLCP/AAAfumha6/6jjvZFZy3cYUQAg+xwUBfbkM0vZq/lKDmPgAAAACUBnnp0R/8AADOXOjVTh/1D0DAA++IRaHLsvYdNVxWAzsZoW1WZAAAAGQGefGpH/wAAM38GJR5UvdGEGmmccsNj5XcAAAA2QZphSahBbJlMCEf//eEAAAX0c5MYhXS9lzgBNL52IStUBGVY+WV5Jj3f9N3jahEdXyUbjStIAAAAN0Gen0UVLCP/AAAfwRPgCJAhw/QO20ndgXY9brCF6scLkYNvDKY0pPVmpR84nSVfXydeZIuNRAwAAAAaAZ6+dEf/AAAzl7zD+dMVMGuGbJsUxx9WFxEAAAAuAZ6gakf/AAAzkn5WNORIl4kVPMCCeETtzyxT7Wh2ABOx+h1ID3g/1qnZKGCygAAAAFtBmqNJqEFsmUwUTCf//fEAAAMDtE8h8AQV1Vf5hkXuT3paLJIgsvatj5K5KUzh4rrxMbD10hds32VFAKXmJusA1ayxmpDCWYcmePGO1K5SYuHafAqcIvm8Z2mhAAAAKQGewmpH/wAAM5JfkIIK9DvlTW+adFlghs6/Yb+DkAD78IIEPPVhk5jQAAAAVEGax0nhClJlMCE//fEAAAMDtE1MexyXFxyzbVg2zgBUjIpDCumk0nvBHH9R9qv+nH2oWQkKILBnjMb4F7H8EPhhFkxer+ZTrbwJLE+KkaWW3xgWTwAAAD5BnuVFNEwj/wAAH798VEekMAJ1fOv9lqoSs8h8Ukp+sqLNKRaFgKQYMUgGQBxGwga7tVXATm02EEr3ikxFDwAAAD4BnwR0R/8AADNXfcjbh2gAWP9vfU1r2taPblpzQIqgomHL9LlFUWIjagzaDR8gVl23u55TcNLUBiliNl4IgwAAACMBnwZqR/8AADJSflaIr4HrdfUXOgA9TffYVJ5Sux3Bgd/QLwAAAEpBmwtJqEFomUwIT//98QAAAwOzgzIQ6dkLPjQ7HeV0oOuFACH1FFs1TbIi6XzVolGnm5slr0Xx9tChlN6ftkc0pW5ng7QJH1128AAAAD5BnylFESwj/wAAH8iCAtf8Pi7JoQufXrkRsru/3A6/W5vwAAWkDW/tdaXD3u6Kx82d9nqqU8TmQV09lD2hcAAAACwBn0h0R/8AADI4ULFCaTHwALRnzZneQufPBBs/Sr+FdRHvTZZS7ssRhS/KXwAAADEBn0pqR/8AADNYGFCDD2IAC56u287aD3Qg2EouwN32xOSP5RcGIEovujIZA/l4I/hOAAAAfUGbT0moQWyZTAhP//3xAAADA8pJsHWvGprZlAb7wpzbNmkjKNmer1zsxbtAPnyVRVpKc2OgEEc/PxZ94L3x5yBni7hhGDRtnNjKKjUZKyOpk529M/KVl6Gd0W0esDv0jBTXSrHOW1Rw6qxxynRVueT+JCcXKRUSrytsQMtYAAAAOUGfbUUVLCP/AAAgt5HwBCD8KcUXyiOP1YI+qeQlPQoQdaTf2dJHA0dsHXFCJj+3GVqbnzXUFXPGqQAAACMBn4x0R/8AADTXVCfFvPjV3xgrgxcKhhJqv0BqYu5qDPw8gQAAABwBn45qR/8AADS+2U0CLKUmy47bAhWu//s0UlbRAAAAiUGbk0moQWyZTAhP//3xAAADA8mFcsAB2sLE4bKtA4w6fAxQcSPbo5AttTp/L/txXjr6VdBQc/6o7HtVdgAQE9cKOXXUW/SMXa4UrlDHTZRSRSmBIeI5At0nHgiMG+ZBeYx7jBWScyn3l+KG+vw+kon6PupM9wqLRG/AqCzKyPJuas4V1Id6WMXYAAAAKkGfsUUVLCP/AAAgrUBWTF+mhAyVg/K249rcAXgnKcDLEvkicXWY0MnVSAAAACMBn9B0R/8AADTXc+7RZ/0sbu7LXsDKbG1nPx2Ce+0uWb9aTwAAACoBn9JqR/8AADTSVILBRzAAs+IXCVS2X/H+Y/GcaNl1O8h16wdDrcaLmcAAAABaQZvXSahBbJlMCE///fEAAAMDzjMyo32hgH/JMOK6pPmaibgAbFhL8xlRlwLcsYovsh8wbSXtvf7hnK72v4q+QJkJ/Efcbw8CZgHqDtq2T/Jc+FazoPQkvpR8AAAAREGf9UUVLCP/AAAgvHpb8gik2AASop1WXicV5SG3X7L5+hAljYTUubUn4dtym3XdGuHl8W8dTaR0+ObgCtC4/3V7HUzhAAAAIAGeFHRH/wAAM3gDZN59qvNm23I+W3pRko2BKzQ+kD+AAAAAKAGeFmpH/wAANL8d7Cpr0KZCwi9ISHAAJap1LOALOTXMjNdLZD2NfN0AAAA+QZobSahBbJlMCE///fEAAAMDykm6GBDjllhOCyEBHPRoSxQNFRppqSR4re8WVdko+mJOMqUQoABcdYsB31kAAAAvQZ45RRUsI/8AACC8h4o3DLbB3+MlE6LUcf0d187V31YAE4dNb+0eRnWWe0DUGk4AAAAqAZ5YdEf/AAAyOFe8heXnuAC3gEIxzvgJZrtHyCSCcLgYJ2AAmCTZMbVTAAAAGAGeWmpH/wAANL7ZTQItMT5Lx1OYEL2ikgAAADFBml9JqEFsmUwIT//98QAAAwPJgxn2S0J2cU3syKa2X5jeC9SRemIjtRLRGkRDS0n5AAAAIkGefUUVLCP/AAAgr+tPrP8m1CzCIKAoVmtVbdvOBIx1YB8AAAAkAZ6cdEf/AAA02DNn1EKABbgBojctwa5g8whetbmHj4G0QKmAAAAAGwGenmpH/wAAMlJ4oe/5V2sXiD8jPFHc4H8VJwAAAChBmoNJqEFsmUwIT//98QAAAwPMKSLVJ/AT9dExgmcSOFP9AXAdAHVBAAAALkGeoUUVLCP/AAAgtnSSgBGRcjjGewezsnwbuUtVKrDQzrmOQMcUc7w4os6UuZUAAAAjAZ7AdEf/AAA0rlkogA4pUK/BDs1ryP6C7UY9amBBdyogIGEAAAASAZ7Cakf/AAA0vtpeA5KuwugYAAAAWEGax0moQWyZTAhP//3xAAADA8mDi8AqfLfrKUFQjP+jy1gSy7oNiJIbUsoJT3c0Kl7CmzCWaGgQ55mPMvMkSh29QSOR4Vft8pxYh7cH+jXceNs0dvMRHmkAAAAkQZ7lRRUsI/8AACC7FPy04KvFJxAjwPanpCwvzFIRelddGw8hAAAAKgGfBHRH/wAANMdFM7+g4AJYp+bM7GE4OfO4975ufRpJGadXuxMQJQUK9QAAACEBnwZqR/8AADTSeR5BquIyR1ekA4vJSkMrKrM3MLvGPScAAABBQZsLSahBbJlMCE///fEAAAMDy6c0wAXSi+lRz9u1DxRrHNvdohvZGSo80ZCq9zrb9r71UatczpkApZMb3SJEkgYAAAA9QZ8pRRUsI/8AACC8h5tuLJFjAB7dZYHmey9n2YPDBKOyN1HgWcpjkMbdBHe4V8tOo2/7N/G74PmaNygNUgAAACgBn0h0R/8AAAduvjtPmFJxYcAC6gD4jTOhkUdJ9IvpNVeIiqpd28zhAAAAIAGfSmpH/wAAM39uHXqJ5Cd8rJ16bQy0WVj42BqvUtFwAAAATkGbT0moQWyZTAhP//3xAAADA8t3za4ZHtRcaExEtYJ+eayuIexIBrx33L7DYzY7VIpxSouR93kwOogTo4DRthNqQOpXKwgyrFWdIU72gAAAAC5Bn21FFSwj/wAAILyHm36GAAuouRPkRazPGnOXVgek+a/vSdDgizjfxp/hKqpBAAAAFgGfjHRH/wAAE+8vtFTSqfnkRoDBAtsAAAAtAZ+Oakf/AAA00lS0MPUxEHa/Z6A8kACdsX5rS4rL8eGc6tLAW9MIOzMbTmk5AAAATUGbk0moQWyZTAhX//44QAABknc6k9WAarygpAXynfEki1GmIrCzbjCfynfmeWu5bCj+CNfelqduJYFAX5vmxJRULupS56sTUohvSoOAAAAAIUGfsUUVLCP/AAAgvIebbiuZWTKSfA6FFp2wlk8p74aTgAAAABYBn9B0R/8AAAMADtQ+sKi3Gyly7lQJAAAAJgGf0mpH/wAAM5J+XxwNNx73EsAGeICn0H/gojnI+Hz9IgiFfw/gAAAAb0Gb10moQWyZTAhP//3xAAADA8pNZoIB8Tp4iJ16zUbgYpJO50P1O6V6IlLbmB8Y6hNPWUJ6LKOBfcFHCqW9LCahQR6jeJEFLS+1jZmu3nEod89bhpwl6dI/oDjSzz2NtfibjivaQ8w+Hpk1RZVg0AAAADRBn/VFFSwj/wAAIDw8Jwrmtdj2C0l9GImMbvAj1q+zcci6lHkMA+Z4SVMCChi9LwoEg2pPAAAALAGeFHRH/wAANKabOXD6AE/oe0wdpP4u5xZx/JdaQyv2Vlq9SsDvDtQub1IPAAAAHQGeFmpH/wAANL8GNB9audp3njeWiqKRdHsRPyD/AAAAM0GaGUmoQWyZTBRMI//94QAAAwDdivpFgdevR0FaAFr5Vv6uComUtcmhGO98DEiAi6rAgQAAABsBnjhqR/8AADOeQXjuDFJbIw+tiSc7giqo4WAAAAI1ZYiCAAv//vau/MsrRwuVLh1Zn7TqDWxYhqt8KG+5XwAAAwAAAwAAAwJRiSUyV4RVE2QAAAMBcwBCwkoeQeYiYqxUDQ8vg5lC0BGVRsGXYji6COOW4Lrhx6fV87LofLASu1Ek3Ox1PcQP8T9fQpE5HAWbosU5v6BC1yGtYYkq39pX4/q3wVD/RHXbTdOnosgUiS/4e4iu6QXZl8GMnHSfkv2CiAz6WIOvLw02knnr8VpMpbgBnEcUWJK+mQcQOKH9N4AlznMUBYXstsmSN5sIhbGhrWETnlT5d2A5N7NINsjZX/QgWdfGkx0Bl7drJIMTuS51xDzLg/ZPan8wyDDrJk1xHSanQQDm4GabCx85JhN8z/IYxHQzOFUxMvwCEMCRXMPWuvMbQdVEG/KBjmNO0G25WGMwOhAvSyUbTqWguQuz8ENpLp4A8WGrHJhv5X4JPTlD0JfGpBkNZFZJruiyt4MYGwsN9Evv1VDQZb35aVFFW1/CgnsoWvRosIBK8lZwrdLm9PMa/saDy0ELhCkSTHB8zLnnPpy4fyA4AgVC/o0LOJTT5Pmiu/CNdXvRIL1vPiUnQ2+GvRNYhIPeidhj/W+IMrP7B6PdwUEvloYVZ3s51wTg8L5zscGRpc2QJsUXo/V+8Fk3RUGPlbDrCuJM0PpkAG9RjaTMBKlq78kcd21ERHeLa+m50w0ui78J6a6aaBBF/0xZSIt/oAiVDQQFUAAAAwAAssUIAAAjgAuwAAADAABZwQAAAHxBmiRsQv/+jLAAAGercgBB6LXYTqwiOyB3OWILbeNSeyEwakbmS3PexNhx8SlwIedPv699j3qHgPy0cRPHCyB8D4/6NJ6C+g+VWLxKtWHJ7kxcKax2lvvQtNae2/IUB3jfpGGBXWx3Q7T5TNJaP7gQ9EVeyD2Jtq2feJFgAAAAQUGeQniEfwAAILAqO4gAjHvFHK3Lv4QZFiXWrPFZweCvTFogUf0TPB23URa4hWRppHL70P5PCRM4NfRHYItClG8hAAAALgGeYXRH/wAANNeuHZi2cAMQY7RpIvbHjR3SajQASwh65QTtzbvssDD0/fk7Y0EAAAAtAZ5jakf/AAAT8qNlPlm/hwFnJjtQAAlj80R4MvKgvUnVsUja/h4xeyuDl2mAAAAAkEGaaEmoQWiZTAhf//6MsAAAZ9GMMOsAmr+fYOdq1BT/zNFYWpDBKA/8ove2P2uAM0hapb2JDS7vHmMGpvm6tKj5Jw/KNDx5TI4SOeFCPYlGW92f7CF5rlX/X/G1IfC0Gnvjx2JKw2Z0Iw+O+XPvZ7TVFPEfQYo75Su7bTVnBsLYvSV/6CJvaZ75Z/mkWnuTgAAAAC9BnoZFESwj/wAAILyHmVLY1Ye/iUYE8YYuSmesXLs7FLw9GHrzvUEcmLU3lvjlJwAAAC8BnqV0R/8AABPyQlg49lWp8zZoSVjpd8KGGDjrqHTACZTWY+Bq9YB6oef0/CB9TAAAACoBnqdqR/8AADTSVLBLuU72qbVsj7N3fPXmPpQ4wAD1TH0b6w2QySFhGLEAAAA/QZqsSahBbJlMCF///oywAABn4fA3HAJqaRKxQTIr7r3OsOcbEwtTDwoVlhO/Tot6OYeT6Bja6Fmaq8KLVxfwAAAAN0GeykUVLCP/AAAgsAxaImME9ycjJzlRL0oQS3yjg6LYX3QAAh+O2MBtxj7k77/UhZb8K73HycEAAAAiAZ7pdEf/AAA0zl6P+oGA1Oah9LW1urayWHEQebFD9dDOpwAAACEBnutqR/8AADOSeSNC0Z26N6vSjvgkyqtlnGjAjr9sFtEAAABWQZrwSahBbJlMCFf//jhAAAGRRivYA2qokWU51CMSaKKGT8SfvQ++PUisyYnIx1lfsrJhGM7RhRkoB35fLCxwk5kTXMvshcDFp0aPeqEpCnN9Ebeu8nEAAAAyQZ8ORRUsI/8AACC8jiv+BERuLfThrK3clcsOAAWAIGDlyfunJOOhUaec+IjHB44eFJwAAAAcAZ8tdEf/AAAzeFCyT+bRD24IBWY/60WEUgBQQAAAACIBny9qR/8AADTSVLPiGabTIJgnvnuoLbCWulEiQ8ySEsP9AAAASkGbM0moQWyZTAhP//3xAAADAXVJw+pKsLadpcLlbABDdMYmi8GQ6eWCx9EJTVzz/t0x32ksv5a9/kaovxqn99ywtqvjFa3reCk4AAAASkGfUUUVLCP/AAAfzchRF6r/3WA8SMIva6Mudn0JUOE6UtQwGhJ7obc0Z90UconuswC2ij5BxTOhn18apnHVeQplcsvi9oSqjKTgAAAAMwGfcmpH/wAAM5J+SFgbowhuFJvJhOnTbQFmVIAT7xnDmAAPa0P20L+n126t1YSVstm8gQAAAHRBm3dJqEFsmUwIV//+OEAAAZIecuUghvroqbYQA0v2Ly5j0CPOOtFiS1fm5Aj2MlKiNNYnhvTgtzkWo87ot/7ZTulzkUnmpUQPjX9/xec6EH33IQOqEGpFbcg9ceD6pu7dP6e6j9X/OL5Q/m1InTRwHqOv8QAAADlBn5VFFSwj/wAAILAMVr8zTWMzaY+6qsWWe8qfHbqDY/ekrI+ZAkVwAhOUTcopQQ5Ptiw+SwZs3kEAAAAgAZ+0dEf/AAA0uANIe4fBwhD16y4uX4Ppj0zubs46d1MAAAAcAZ+2akf/AAAzknp+J73xiFTvxSNHUixJbSDUgAAAAEdBm7tJqEFsmUwIR//94QAABhx4Q+/oOKXPHh5U7eDveGVoLeAt54ATWy/igV4XFMcVo1LD4gHXIp9SvQ1FZXwbZrD0pUn6oQAAAERBn9lFFSwj/wAAIDxOVFavuhU8R6LfAKGCoRl3lcggfvy1EP7AA5OznfrLHl54i4lC8/uqXKMOs0AkhnWCkfkfEvfJwAAAABwBn/h0R/8AADTWblpY526vhO08sMQo12jewB/gAAAARQGf+mpH/wAANNJzwqSwQg4Pc0RwAFwwDA7wHQ9xkJyRL4gLVCv9MI2BntGUAgD62kruN4bR5PfanN4Y+T26soeDYt0PIQAAAGJBm/1JqEFsmUwUTCv//jhAAAGSHnLobkVwCaAq9i02W/AA8HrF9fAiRzq6vYDAVxcKGXWMPIYkJ+noqSpm3TBg+jHatCAFHjXyLPVLtm0vvgfY/oNokgAee9tCoqrqqLh24AAAADYBnhxqR/8AADTSdVzoZgx34gAsEM/RTAzVfqcan2f+W6oGeP3TLBFS5fkIBM/P93HbdhJaJOEAAACaQZoBSeEKUmUwIR/94QAABiGjPZpzEBQGcDNSk/Er4BfZivJQ9B4aqd4aMfR8kLaMAMw4pnr0WHgHv/JCSB/+KgNujPNlhqy6pV+EX6Z656cM7+2HYkOtj5P7JhYOvDgpvHOQcgpB7dyLbO/gY5FuSHv0aoidwWk4BWeOR5Br7gWZwm7TFG+uUpoPOu5ekd9Cx4o4nsGgvt5YeQAAAFZBnj9FNEwj/wAAILFl4AiY2KJ1EIKBIoO4RTpARZB9RGm5xE8gjBkBNUY9FeAqDh5FeXetgpirrqZDMAddEWVdGPqMtGYKLnfAujn88Be5G4AWnTaTgAAAACYBnl50R/8AADS4A02H0mfLFtlvPEaX7esIY1qf/5/jZtP5jevpOQAAACQBnkBqR/8AADTQrIsYzk2IabgjbHJSIHvBSDB1oE/yaa+7mcAAAABkQZpESahBaJlMCE///fEAAAMDyYYRoEUlU8bVRlhkACD6ebZlj9EVCV5el+TF+/eutVtazHDnhk9IjIhUkmNaGO4CefGGp47egvllPdyC802GH4hRzqCLWBwbVlJECUXI9pt3VQAAACxBnmJFESwj/wAAIDxRJdOohEPIAEtVkEepihFcT0ycpJ+zVBwNnL4V7U1hLwAAACYBnoNqR/8AADTSQzcS8qFz3hjt/qKYUTKhnsjtwgAGpHEtawDpgAAAAEpBmohJqEFsmUwIT//98QAAAwPLeZfsBpKlgNPvw3G9lqcaoL9JwYPwSNjxXsjVcNjO6McgOBUNAox1mKZ7rUGukd5UmHiqfw8j4AAAADNBnqZFFSwj/wAAIK/rnG842AAbbd3yAULbNVFslTjKLyXiwkYA9PBadLZkzgS1tYGVjE0AAAAiAZ7FdEf/AAA013S/4BRNk3vNpLiOlHJKYrp/HLXGO99ZUAAAACMBnsdqR/8AAAMC15hLQvQrl78zeiJCisRLpgT+V8bJUG5qkQAAAFVBmsxJqEFsmUwIX//+jLAAAGeX+OQAtZ64TKSSVsCHd8Ytu1kruHUEbQ6SCKdlqfzKrn6HwLTIP2+B2XnWedoZbBFsckbyPPCTXlXPjhEumKo2Ub18AAAAJkGe6kUVLCP/AAAgrRpfe03tfvqJ0ZXjO4J9PTkzcd+IsoPsv6qRAAAAIQGfCXRH/wAANNe03HCN4tmB7BrbD3L6O4SzKLLEQ5pqkQAAABcBnwtqR/8AADS+2jDcnLDUdWTdXD6DewAAAIpBmxBJqEFsmUwIX//+jLAAAGfSDjcBQz1wmSf+Zyppctf88AtK3svI4hMnXCTSwW8UmfhNfd4SFBqxt5KgsQTZj1R6PTbFUseQpFQxj/ia6o13hI2+YBNVwqVetinmAIgAtaS05cCKcFmx6DIA0gDnv1C+q+hq5LBLJslFLMW6MU+b9YaEO8HQmtEAAAA2QZ8uRRUsI/8AACC8h5tuLe6UGdujKhcrfeqZOV/McIvmgA1IGvukB8nFSPawMW98/h7GQ1SAAAAAGgGfTXRH/wAABz4oeZLHx//ckaNJlPYkRQfwAAAAJAGfT2pH/wAANKyHyPM3HW4wp+2v1pQDYcLwLkTSHJ1Hbz8zgQAAAF1Bm1RJqEFsmUwIX//+jLAAACcG/CpZ0bQnpW1QAi/cB/a2MhHg/Fd3s7C2BUc9rFHTz8gQHte3918zH+JpH5yzv5Q87fdM4+LiHtr3hNjig29R6KJHzLuYl305Ik4AAAAhQZ9yRRUsI/8AAAyVgGl1G6g2QJj8+3cgKGiDbhvOtUfAAAAALgGfkXRH/wAAM7RAgBYHNPPiGwVRh2Xrnte1L6S7/8SSu9iFk1DwJjdJOCPEQqsAAAAnAZ+Takf/AAA0ulC4URuzRBGxaIynPOrOTPABB1Q34PP2ieSf3j/BAAAAe0GbmEmoQWyZTAhX//44QAAAltFUkxewAXQNSX65Sreb1Cq6tRxA6M/17zXzSrNdm+k2pTvFqlARIOo8l6ZZzwgYxpdoAqnnTXX3dabMrS2RSf+qP9T/4CQAypjrByW5/Hh3gNnpOvKzhIgKD3M7ocgInxZ1Gp/QP9NRgQAAAD9Bn7ZFFSwj/wAADJMrDKky/pqgRn1R+5Jmyh0MfxhPRlOg77szZEAJq7xdcws+P3XVMqTG636jjqP2XTyWYeQAAAAxAZ/VdEf/AAAT4+bSz++Jf3TzdG5TBPIryGsIC769ABAVOjVtOWg9XDiKvEgXTKkHgAAAACUBn9dqR/8AABPpA7uipQocnYJP/lScI28gyxsD8ZHw2eqwbQ8hAAAAWEGb3EmoQWyZTAhP//3xAAADA8t3za4Y/oel+wACv0rDuCesxwAF0FFodTZHUeCqQ2l4pR28H7NXWsXi7LIN8tmH23jW/7VqFryERioUOFdcA75y7yEwzQwAAAA6QZ/6RRUsI/8AACC8h5jQhqAVqVWQTNnTWW8TWHeOwzC38Y16EHjCDPpNHAO6DhZHkHiyllB+dqnSBgAAAC4Bnhl0R/8AABPyqqW5C+QVMyXkDTx66yvmVseuzByAFqwhkxMCItPs3di27JixAAAAMQGeG2pH/wAANNJZpTEFLqEeP3w3SZoDFnXW8B0Aue9/KACIOhT8mbqarNacKIyH86YAAABzQZoASahBbJlMCEf//eEAAAYd+pxyVwqGZg4OSwiPYgIuD5uJNgqj5fQK7GBUqQvAbfP8pmZOxsmuFoLpGazvLsQqnQ+9fbr2VuKZ6Uu1TZ2luo7SDoSmdvLk+bwl318jAKJd++t4PUV9eubn+w7rigXygQAAAD9Bnj5FFSwj/wAAILychqCJhaY6rYHR3uCHLJybMlVUu2X/zbiQx0+XuoAD+dSaotUVN9u5NQTAJ/36GqGVmk8AAAAnAZ5ddEf/AAAT4UmnWMT3CMRO6CoJeZy23MexfvgM1Y9PUVXJIuV6AAAALwGeX2pH/wAANNJ2B7m77ZkbxehTAcn3x13Tf5UPsAAaojZ6yhli8Uww84TPEHrbAAAAWkGaQ0moQWyZTAhP//3xAAADA8y3YdYw74q5AUuWkgAHoj7xdnetD4YkmVZwSbIardOs++pfqmwLWoNfe7ai0TCL01tL8ByI4dlGBJbkDf/a6J9Q8fxiQzd5AwAAADlBnmFFFSwj/wAAILyHmY+YFoMsKr7efyFYrfrAmpK0Bh+wqpXoEumjmwAgVvBs3y1xSYpJuGMfm4EAAAAlAZ6Cakf/AAA0vto2EHnMLMxi03yOW7Zk9aO5xReKqH5H6yopAwAAAHpBmodJqEFsmUwIT//98QAAAwPJdP7gBazqeIKnkq3XoakFUctaO2Wp4IDekEf3vCTH8TCoJENuo9lIET6m1uSCg7EaBs78ojPonEHAzNr85cyY2WSTwe5QOud/EGkfKE47avtroBFhzDnoTlUtvyBJldTLaXjvHsPQcAAAADBBnqVFFSwj/wAAIK1CNotL00tQAsWEP83RVtHzerrMm72KIQZQX6QCNHFW8fuaQMAAAAAxAZ7EdEf/AAA0uEw9qLyuE3iX2GkD6JNEzlVDnptCq+r/Qc1CJ9gIATIz5qQUXREgYQAAADABnsZqR/8AADTddgD4SAbQgk5cz3gxlJH/Ob/bFUryIwq23J4/noAJrhojNbaQ2YAAAABXQZrLSahBbJlMCEf//eEAAAYeZDSXdknkHIST46rQIVQM6DorqPxXEvqEFPPJ7+yivZGFCVyLwjUzbyyTfvU7Qtgo+fajvS+WmH4WvktUe4tbWS5kWi+BAAAAREGe6UUVLCP/AAAgxInSYwPnKWceUVom/rzgfB1uXlgAF1NV59FDdEsTSrhuwJhQlp2jYa0fnsa0RSy7Aj7JbIKilE1SAAAANQGfCHRH/wAANMdFctUeB2eninIeVrL98wXZSKHdhIVGMkoAH8G99Kac2/JINuL0L+uKy/mdAAAANgGfCmpH/wAANNJ2A36BZZ435ZmjYWK0UVsRPXKKvyYZBF3sEkqxJhL3AA+/+KCi0eXE7CwuZwAAAHpBmw9JqEFsmUwIR//94QAABhxcEXgGpSK83daTB8nUk/pP24ajjZd6RXzTjcWP4ENvgumEC2ZWHs/b/WUwEfAlfrYqKmNyOgwOWPq5c7d/alWwTDft1sGLzQe2eLZH+82FnsXwOnjtUMzs4+syD9Kwg3e2RPogNmbAsQAAAD9Bny1FFSwj/wAAILsVo9Ns3tSCzGcEq1zfbzOZsyaovmqNgACalIFqTPgmyx0+8Y+luaI3GwOr9xxXzgUrOZ0AAAAtAZ9MdEf/AAA0uFccqUJM2Um5ECwNT6YJNGeLHZ1PekfMfFRBzeaCSDQ3Jgr0AAAAPwGfTmpH/wAANNA4HH/vfJa6GWJjooALnPN+nG6TsriGgnLaaLo2PfKSnU/qYaq+ZmER8V3rVtLO2iGatdjmcAAAAF1Bm1NJqEFsmUwIR//94QAABkQ/leD6VHiC+RYODhLkN91YbGdugATKxFJEnQhgXo1l7e90/gow3Ia+BhOEe0KUZQehNvWPlixVdZEHG+SLMxRWol11MmD0pMiK9r8AAABZQZ9xRRUsI/8AACGsxeeATVB3kmKC8ybrxt89llsM1rKCq80eo+qVcLeYjR1k/iVVcQlzkB08q5vuO9zHeEEnNebycsQNnyc2/6k/8otvoW0ZXBG4H+8ik4AAAAA4AZ+QdEf/AAA2F1OmzOxQmF9odQCBaZ5U7rMgnhvbtAAuMmF73OOus4R0gtszKqBQJ9gd2DR2V6AAAAAuAZ+Sakf/AAA1+lC4UOkOp2G9C9maWp+2inRGCyjKruvwCsAD+YW/myItZ8ylbQAAAGlBm5dJqEFsmUwIR//94QAABkZmAbd9h2N3qUUh5+JH5h+E7ckLnZZtijB7KFhXec1R/69sYS0gBtsqZnpkc2Ef2w6dADSsXtUYARCGnsqjt1i0w/cVdgXGA6zfvireosCc2sAmf4sNzoEAAABAQZ+1RRUsI/8AACHEYWnqbVAES7HbGbw2Dl/mQAW8OcxVDQ4l0zu9yyh1sDtYdkC5F9hkvIArmY4cmpyDx3Q8nQAAADsBn9R0R/8AADYTso9SAES//nCJ0whFIatUDDEbExwwhkZdJ7vOD7mZJjFa2GQRwNLVyZlec+OZuSYA8AAAADUBn9ZqR/8AADX6ULhE2+NHVPfbar8bhFbyWwIZUPk9yEb1NYUypWXdLeWoibIMNiQGHia/gAAAAIdBm9tJqEFsmUwIR//94QAABkgR++eI9Qyne7o8FILYSBrH0IO4mAFAGN1kPLEgTKqJxX54Dx7zRCh60d+4XTXm/+FDwJsi99UjnPZn3ewWpvXY3t1C2J0FlwfQODnryemDrFqSK2c2hHJ7TwcesPSGnhUdPIrxgqw4GdcpofefJCy4vzM/2JcAAABIQZ/5RRUsI/8AACGsbVS0SPVENToyj6lYdrz1kBKT+mUN6NcDvMMJsF4vAARmraPLwOX1JYeEVCy0zwzEQyXcxPwz3rTPU6PgAAAAOAGeGHRH/wAANhQSWDjHUiEykDsMrqHuxr77Q8IigPAwJOYe3EhTAJBlUADYbMB8UFFF+KQsq8OIAAAAMgGeGmpH/wAANfpQuEx3C5vv4sOmzZnJDD9dEAsM1y41icMaLbJd2y58CAI6KtbDqPuBAAAAc0GaH0moQWyZTAhH//3hAAAGSBQp6FgAT2K7jPk+ZAyOi6jWmHGcAiqY21b3ImUwBh799Dx6w+nJYqnLSDm5SEI2Sw+cQOdvw+wY+xE1CFXZ7K3kvGYs6oJxERQZoJWUD/Di1hv5QE0uA+JV5RRKrJRau/gAAABPQZ49RRUsI/8AACK8Z3jv13emOm4NKhxDU47C8Huf43oER9bGbNnfNhbjU4ASdIru8/rVaTB0aufPWDOVnVzAC1ImpvX6v//6kW0XWLbU+QAAADMBnlx0R/8AADXnECFESIcsYZ6gsoHQaeogTCh7rbHyg1JgBSHABtOb36Vfm7oHXFwvR8EAAAA6AZ5eakf/AAA3Uj424Dzc+MER4cVR8nlvUNGpKZEclnIuxrZeYw1+0ACGs8+QFFiWWU9JI+rPzuanwAAAAG9BmkNJqEFsmUwIT//98QAAAwP1aH+562ZFo8I83NaBKnwAmOisy7R3EznynzPphuduSN0fvF25YD82wwAeFB5+JfBGuyX+045v5j287jlD+0ehBZVfJi1t6L22avYhAD/Rwh2VE1wMRW3eR4VerLkAAABKQZ5hRRUsI/8AACLFEydxVKYrA8BvO1v2JN/hbw761ztAOondgcAStEAA426DatLI/qRl03cMi3pbHJkY4B8v2MPZRfiOj0PHR8AAAAAtAZ6AdEf/AAA3OCSUDuIrdazG5idYJICTHEQRJcGNNV4DdoMslQlbI34le+XxAAAAOgGegmpH/wAAN1A2We94DIQOTCu7MmMF7tmZt4q4jgvyvJRBf2oVtaAYLIQyRw3kgA/tu4saVqrvy+EAAABpQZqHSahBbJlMCE///fEAAAMD9kfgK0OsB9sB5uNqnsT8IYJBrJjdI6V84/ZnpYsDmbIwvwhcqBLMO6uvOIQtfTGZfXuWoblwxDsCrlgaoLs7k2Cr9g6BbWpQoHtF+ZjF+f0hclnFuWJwAAAARUGepUUVLCP/AAAiq4UX7mqAK3jZRrYoQJTCgjikCpoS0T2u/9cBkmEk+bPpt740nlqnq2g3lWj3HMVHdyVr/MACcX5WYAAAADMBnsR0R/8AADc1SqtusMLX2/iNcmJBzuMDoAUTU5DHqDY6hKsCu4Qcl8A8JQdCaoraNPkAAAA2AZ7Gakf/AAA3ON2ny2wfXQ6iZFgMs9Wgi1Tt+EBIS2ZIEo1k6qYdhzWkKBDgJboqDjgCd2OAAAAAc0Gay0moQWyZTAhP//3xAAADA/VsKjSRZhynXAQAImAePr5EfpOYvOXYNiJ+UjBqWrmGMkDQmS+ejc1FN2dvI8ZiENPmRTZgGC2xEwrzVtn1B+rcgZ1Bx/4jHqkWkeinlhM05RxnkPb+7fdliLTKXTXw1fsAAABBQZ7pRRUsI/8AACKrhUN5VgaA7lgqPlPgADjOrQRViRxcZ3Sm4HZd/lJ1HQf8JC164dMGgnWQwrDRbxVyleM8xcAAAAAzAZ8IdEf/AAA3GMup8QnbBAmnp66W4SvOpJWXg2xqV6V6Q3F7wfpe1vgMJ0ImdpJQb0PJAAAANgGfCmpH/wAANzjeVzEO0uD+ChLrGQjME5otrM8IhdMlW7rCRe0R5Ye4XtcYaud60SqEZgs3oQAAAHxBmw9JqEFsmUwIR//94QAABpROK2QCk6kApJsyLzQapVtWcdG7dsduDgpXgucAOcIEBys3yJcKE5uTbgK71DfO0PUx1jh3O0JqhmNbBzsEQHM/7t4qWHXCseLzvW/AcfsQ9UoNUdPfGX6QtREz8RHa6Ss+5DauY2N3vfWLAAAASEGfLUUVLCP/AAAjvioXYFu0Xchn+9CwRh5eoABEMXTrcHJWr96T0i5SkGbaVnLxKBnbohmzFHvDw9MyytgK5lngETC8oLTi4QAAADwBn0x0R/8AADdS8u9zHNGZMMWe7JzhGOCdTDn+CCVLh4SJM97NiSNHxY6d6zCpJGmlLP5UR4Jn6yUUwFwAAAA4AZ9Oakf/AAA4k7gfEr90i6brGvATe07Yq3cbkJ677BrX2YABbGN83SJFvLMkkF7W/7BPZbT9H70AAABuQZtRSahBbJlMFEwn//3xAAAEFLYK6K99GPio6OE7mF74Ze02FRol91wYv6rZPtOR+A/wGYCh8J+baEmIzACWROsh1a19qJ1bkxCNKfxyVPGo76dmD79UBc5qv/u2UHhgu8I5u6JzA4XDyPHI/KEAAABGAZ9wakf/AAA4rL5xnNX+AJ8vH6K5WCH+3zlxv2Vzy0DYlbnhkweWWM90QQqC0OjPtgcAE0s0pbsojG65R/WKPWolrrxvQAAAAG9Bm3VJ4QpSZTAhf/6MsAAAbyj/I/zgAug0IGdXUG+JZ8fqYvvz64CUJWRMLwxi5mLDJGoUo+psSNm+xXiFEaDn/62qTDyW+8kH5ibDoSM9VrzGHLOmfhukb5yw5Txu8Csz7LGkoEt/GpTIL/+OyRYAAABNQZ+TRTRMI/8AACOrzgtqW9w4EUUmojQAJ7wgNfR3a3q267tt3z8PC/TW2vrrJsMcJWueEncfUl1zsvCJzZGgQUaFz3DQ1lW+a0gGVPkAAAA8AZ+ydEf/AAA4sEuCuiAPZczQH2RomZKv+trMu4zl3s9eAoyDUOmfZhBEjx1RsylhABK+he6Nv4fbXzgfAAAANwGftGpH/wAAOKy+aHV6tCacPKYsvQAor9BstKn5A9HFWwx+AEBVDBF62eFKgHLe80h47cRfTMEAAABnQZu4SahBaJlMCFf//jhAAAGvPDYgYAONOxAoPK5UPwQJHhhWmCwTtgmvfsONENMvXQIejUXsn99eRvfcoB0Oqg5Gm6acs+Wrh56GtdzvHNw5OMK3b/MSvwKBKaN6ww3KrD1H9msGgAAAAD9Bn9ZFESwj/wAAI8RzNi8LEAMFziRqNQyz/I6P6ayPYAAIhaoFa1rX5Z5iudXER7Trz4Ixn6294lsGqUR9xfAAAAAqAZ/3akf/AAA36RnhQ+LchbL7SrjlC76Hu4vgNnsudSasfNab0gv3WmZBAAAAN0Gb/EmoQWyZTAj//IQAABnfRROXsAvtgbZ6DW8iWqMChZ9/RTlM6r+YPBGiAJGw0478QjbE8lAAAAA5QZ4aRRUsI/8AACPAy5VsgAsEeEnPfyzneaXXsp66DzT8sXuefLpN3NRS+CMti1IA2yMB3c3UuowIAAAANAGeOXRH/wAAOI0FfJNEznATLn3NSbmSqvubvyFhizcsG107aYAAPXtdaXlki35P4JeAwIEAAAAmAZ47akf/AAA4rL55XmLz1K2qGfpG55A+QV5EgEi5jDwV43dr4DgAABRfbW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAHUwAAQAAAQAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAE4l0cmFrAAAAXHRraGQAAAADAAAAAAAAAAAAAAABAAAAAAAAHUwAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAlgAAAGQAAAAAAAkZWR0cwAAABxlbHN0AAAAAAAAAAEAAB1MAAACAAABAAAAABMBbWRpYQAAACBtZGhkAAAAAAAAAAAAAAAAAAAyAAABdwBVxAAAAAAALWhkbHIAAAAAAAAAAHZpZGUAAAAAAAAAAAAAAABWaWRlb0hhbmRsZXIAAAASrG1pbmYAAAAUdm1oZAAAAAEAAAAAAAAAAAAAACRkaW5mAAAAHGRyZWYAAAAAAAAAAQAAAAx1cmwgAAAAAQAAEmxzdGJsAAAAnHN0c2QAAAAAAAAAAQAAAIxhdmMxAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAlgBkABIAAAASAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGP//AAAANmF2Y0MBZAAf/+EAGWdkAB+s2UCYM+XhAAADAAEAAAMAZA8YMZYBAAZo6+PLIsD9+PgAAAAAGHN0dHMAAAAAAAAAAQAAAXcAAAEAAAAAGHN0c3MAAAAAAAAAAgAAAAEAAAD7AAALeGN0dHMAAAAAAAABbQAAAAEAAAIAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAACAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAAAgAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAQAAAAAAgAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAAAwAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABAAAAAACAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAFAAAAAAEAAAIAAAAAAQAAAAAAAAABAAABAAAAAAEAAAUAAAAAAQAAAgAAAAABAAAAAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAADAAAAAAEAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAABAAAEAAAAAAIAAAEAAAAAAQAABQAAAAABAAACAAAAAAEAAAAAAAAAAQAAAQAAAAAcc3RzYwAAAAAAAAABAAAAAQAAAXcAAAABAAAF8HN0c3oAAAAAAAAAAAAAAXcAAASRAAAAkgAAADkAAAAwAAAALAAAAJsAAABCAAAAqgAAAEsAAABFAAAALQAAAIAAAABRAAAANQAAADoAAAB6AAAARwAAAEEAAAA4AAAAlgAAAFkAAABBAAAANwAAAJEAAABLAAAAPQAAADcAAACRAAAAPgAAAD8AAAA1AAAAcgAAAFUAAAAxAAAAPQAAAF4AAABNAAAAOwAAAD8AAACMAAAATgAAAD4AAABKAAAAfQAAADwAAABtAAAAXQAAAEYAAAA0AAAAgwAAAFwAAAA+AAAAOQAAAIEAAABlAAAAOAAAAEYAAAB2AAAAPwAAADIAAAA5AAAAdwAAAD0AAABAAAAASgAAAHcAAABLAAAAMAAAADgAAACcAAAAQgAAAEIAAACUAAAARAAAAD8AAABCAAAAaQAAADsAAAAyAAAAPgAAAE0AAAAtAAAAJwAAAIkAAABMAAAAPgAAADAAAACnAAAAXgAAAEgAAAB0AAAAXAAAAEEAAAA1AAAAZwAAAGIAAABGAAAANAAAAH0AAAA8AAAAMQAAAH8AAABJAAAAPQAAAIwAAABSAAAAPwAAAE0AAACQAAAAUwAAAD4AAABVAAAAgAAAAHsAAABcAAAARwAAAEUAAAB2AAAARQAAADEAAAA+AAAAfQAAADgAAABcAAAAPgAAAGwAAABJAAAAOAAAADUAAABbAAAAOQAAADYAAAAyAAAAcQAAAFAAAABAAAAALgAAAJsAAAA+AAAANQAAADYAAABaAAAAXwAAADQAAAAzAAAAdwAAAD8AAAByAAAATQAAAEAAAAA4AAAAbAAAAFMAAAA/AAAAQwAAAGMAAAA3AAAAOwAAADwAAAB0AAAAWAAAADwAAAA0AAAAaAAAADgAAAAnAAAAJgAAAGQAAAA6AAAALAAAAHUAAABZAAAAQQAAAC8AAABZAAAANAAAAD0AAAAjAAAAYwAAADIAAAAfAAAAGwAAAFQAAAAwAAAAIwAAAB4AAABJAAAALgAAACkAAAAdAAAAOgAAADsAAAAeAAAAMgAAAF8AAAAtAAAAWAAAAEIAAABCAAAAJwAAAE4AAABCAAAAMAAAADUAAACBAAAAPQAAACcAAAAgAAAAjQAAAC4AAAAnAAAALgAAAF4AAABIAAAAJAAAACwAAABCAAAAMwAAAC4AAAAcAAAANQAAACYAAAAoAAAAHwAAACwAAAAyAAAAJwAAABYAAABcAAAAKAAAAC4AAAAlAAAARQAAAEEAAAAsAAAAJAAAAFIAAAAyAAAAGgAAADEAAABRAAAAJQAAABoAAAAqAAAAcwAAADgAAAAwAAAAIQAAADcAAAAfAAACOQAAAIAAAABFAAAAMgAAADEAAACUAAAAMwAAADMAAAAuAAAAQwAAADsAAAAmAAAAJQAAAFoAAAA2AAAAIAAAACYAAABOAAAATgAAADcAAAB4AAAAPQAAACQAAAAgAAAASwAAAEgAAAAgAAAASQAAAGYAAAA6AAAAngAAAFoAAAAqAAAAKAAAAGgAAAAwAAAAKgAAAE4AAAA3AAAAJgAAACcAAABZAAAAKgAAACUAAAAbAAAAjgAAADoAAAAeAAAAKAAAAGEAAAAlAAAAMgAAACsAAAB/AAAAQwAAADUAAAApAAAAXAAAAD4AAAAyAAAANQAAAHcAAABDAAAAKwAAADMAAABeAAAAPQAAACkAAAB+AAAANAAAADUAAAA0AAAAWwAAAEgAAAA5AAAAOgAAAH4AAABDAAAAMQAAAEMAAABhAAAAXQAAADwAAAAyAAAAbQAAAEQAAAA/AAAAOQAAAIsAAABMAAAAPAAAADYAAAB3AAAAUwAAADcAAAA+AAAAcwAAAE4AAAAxAAAAPgAAAG0AAABJAAAANwAAADoAAAB3AAAARQAAADcAAAA6AAAAgAAAAEwAAABAAAAAPAAAAHIAAABKAAAAcwAAAFEAAABAAAAAOwAAAGsAAABDAAAALgAAADsAAAA9AAAAOAAAACoAAAAUc3RjbwAAAAAAAAABAAAAMAAAAGJ1ZHRhAAAAWm1ldGEAAAAAAAAAIWhkbHIAAAAAAAAAAG1kaXJhcHBsAAAAAAAAAAAAAAAALWlsc3QAAAAlqXRvbwAAAB1kYXRhAAAAAQAAAABMYXZmNTguNTEuMTAx\" type=\"video/mp4\" />\n",
       "                 </video>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Animate learned policy\n",
    "save_dir='./videos/pytorch/actor_critic/'\n",
    "env = make_env(env_name)\n",
    "generate_animation(env, save_dir=save_dir)\n",
    "[filepath] = glob.glob(os.path.join(save_dir, '*.mp4'))\n",
    "display_animation(filepath)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
